<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif; ">
<div>Somehow I missed this email until today. See inline…</div>
<div><br>
</div>
<div>thanks,</div>
<div>Robert</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div>
<div>On 12/24/13 1:31 AM, "Irena Berezovsky" <<a href="mailto:irenab@mellanox.com">irenab@mellanox.com</a>> wrote:</div>
</div>
<div><br>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Please, see inline<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size: 10pt; font-family: Tahoma, sans-serif; ">From:</span></b><span style="font-size: 10pt; font-family: Tahoma, sans-serif; "> Ian Wells [<a href="mailto:ijw.ubuntu@cack.org.uk">mailto:ijw.ubuntu@cack.org.uk</a>]
<br>
<b>Sent:</b> Tuesday, December 24, 2013 1:38 AM<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<br>
<b>Subject:</b> Re: [openstack-dev] [nova] [neutron] Todays' meeting log: PCI pass-through network support<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">On autodiscovery and configuration, we agree that each compute node finds out what it has based on some sort of list of match expressions; we just disagree on where they should live.<br>
<br>
I know we've talked APIs for setting that matching expression, but I would prefer that compute nodes are responsible for their own physical configuration - generally this seems wiser on the grounds that configuring new hardware correctly is a devops problem
and this pushes the problem into the installer, clear devops territory. It also makes the (I think likely) assumption that the config may differ per compute node without having to add more complexity to the API with host aggregates and so on. And it means
that a compute node can start working without consulting the central database or reporting its entire device list back to the central controller.<o:p></o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><i><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">[IrenaB] Totally agree on this. For both auto-discovery and configuration, we need to close the format and content
that is will be available to nova. <o:p></o:p></span></i></b></p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</span>
<div>[Robert] Agreed. With auto-discovery, if all the PCI passthrough networking devices on a compute node are there for guest use, then no whitelist needs to be configured on the node at all. </div>
<div>Also, for deployments that do require explicit whitelist, I'd think that we may go with configuration-based whitelist first, and then if needed, add the API based.</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<div lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><i><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">My concern here if there is a way to provide auto-discovery based on network connectivity (something like what
neutron has i.e. ‘physical_interface_mappings’)</span></i></b></p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</span>
<div>[Robert] Can you elaborate on this? </div>
<div>A specific plugin can have its own configuration defined. What we want to make sure is that the port binding passed between nova and neutron contains sufficient information for a plugin to operate.</div>
<span id="OLK_SRC_BODY_SECTION">
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<div lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><i><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p></o:p></span></i></b></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><i><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">For configuration, maybe worth to provide some reference flow for managing it by installer.
</span></i></b><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal">On PCI groups, I think it is a good idea to have them declared centrally (their name, not their content). Now, I would use config to define them and maybe an API for the tenant to list their names, personally; that's simpler and easier
to implement and doesn't preclude adding an (admin) API in the future. But I don't imagine the list of groups will change frequently so any update API would be very infrequently used, and if someone really feels they want to implement it I'm not going to
stop them.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><i><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">[IrenaB] The issue we need to resolve is nova scheduler taking its decision that satisfies network connectivity <o:p></o:p></span></i></b></p>
<p class="MsoNormal"><br>
On nova boot, I completely agree that we need a new argument to --nic to specify the PCI group of the NIC. The rest of the arguments - I'm wondering if we could perhaps do this in two stages:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">1. Neutron will read those arguments (attachment type, additional stuff like port group where relevant) from the port during an attach and pass relevant information to the plugging driver in Nova<o:p></o:p></p>
<p class="MsoNormal"><b><i><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">[IrenaB] Do you mean via ‘neutron port-create before ‘nova boot’? Hopefully we can close the details during the discussion today.</span></i></b></p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</span>
<div><br>
</div>
<div>[Robert] this has been proposed in the google doc.</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<div lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<div>
<div>
<p class="MsoNormal"><span style="color:#1F497D"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">2. We add a feature to nova so that you can specify other properties in the --nic section line and they're passed straight to the port-create called from within nova.<o:p></o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><i><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">[IrenaB] I like this option. This should also allow to request virtio versus SR-IOV nic. It should be possible
to have both options available on the same Host. </span></i></b></p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</span>
<div>[Robert] Currently, with neutron plugin, the properties contained in the —nic option is directly passed to neutron. However, nova will perform some validation checks by calling neutron. All the properties are organized in the requested_network dictionary.
This means, adding new properties into the requested_network dictionary requires nova code change so that they can be added in the dictionary. It also means all the places that use the dictionary may need to be changed. If I understand correctly, you suggest
that some properties be grouped in a dict field of the requested_network dictionary. It's a good idea. Future expansion of the —nic option, if any, can be made possible by just changing the nova client and the nova code interfacing with the nova client. </div>
<div> </div>
<div>So sounds like that the newly proposed properties in —nic are ok? </div>
<div><br>
</div>
<div>I'd think that we should support nova boot first since it will not alter the existing application flow to create instances. </div>
<div><br>
</div>
<div>Support of port-create with the new properties is an easy task once support for 'nova boot' is done. </div>
<span id="OLK_SRC_BODY_SECTION">
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<div lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">This is not specific to passthrough at all, just a useful general purpose feature. However, it would simplify both the problem and design here, because these parameters, whatever they are, are now entirely
the responsibility of Neutron and Nova's simply transporting them into it. A PCI aware Neutron will presumably understand the attachment type, the port group and so on, or will reject them if they're meaningless to it, and we've even got room for future expansion
without changing Nova or Neutron, just the plugin. We can propose it now and independently, put in a patch and have it ready before we need it. I think anything that helps to clarify and divide the responsibilities of Nova and Neutron will be helpful, because
then we don't end up with too many cross-project-interrelated patches.<o:p></o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><i><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">[IrenaB] +2</span></i></b><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal">I'm going to ignore the allocation problem for now. If a single user can allocate all the NICs in the cluster to himself, we still have a more useful solution than the one now where he can't use them, so it's not the top of our list.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><i><span style="color:#1F497D">[</span></i></b><b><i><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">IrenaB] Agree<o:p></o:p></span></i></b></p>
</div>
<div>
<div>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><span style="font-size: 13.5pt; font-family: Calibri, sans-serif; ">Time seems to be running out for Icehouse. We need to come to agreement ASAP. I will be out from wednesday until after new year. I'm thinking that to move it forward after
the new year, we may need to have the IRC meeting in a daily basis until we reach agreement. This should be one of our new year's resolutions?</span><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; "><o:p></o:p></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Whatever gets it done.<o:p></o:p></p>
<p class="MsoNormal"><b><i><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">[IrenaB] Fine with me. If we reach required decisions today regarding neutron, I can start to dive into the details of SR-IOV mechanism driver
assuming ML2 plugin.</span></i></b></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</span>
<div><br>
</div>
<div>[Robert] Irena, you are welcome to start looking at the ML2 plugin for SR-IOV support now. Let's hope that we will have some kind of resolution soon. The main neutron code also needs to be taken care of in terms of interface changes with nova, database
changes, changes due to the introduction of vnic_type, etc.</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<div lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><b><i><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p> </o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">BR,<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Irena<o:p></o:p></span></i></b></p>
</div>
<div>
<p class="MsoNormal">-- <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Ian.<o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</span>
</body>
</html>