<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">On 2013年10月29日 03:22, Robert Li (baoli)
wrote:<br>
</div>
<blockquote
cite="mid:1705659276F6A540A34DBC19195799841027A7A4@xmb-rcd-x03.cisco.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<div>Hi Irena,</div>
<div><br>
</div>
<div>Thank you very much for your comments. See inline. </div>
<div><br>
</div>
<div>--Robert</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div>
<div>On 10/27/13 3:48 AM, "Irena Berezovsky" <<a
moz-do-not-send="true" href="mailto:irenab@mellanox.com">irenab@mellanox.com</a>>
wrote:</div>
</div>
<div><br>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE"
style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0
0 0 5;">
<div xmlns:v="urn:schemas-microsoft-com:vml"
xmlns:o="urn:schemas-microsoft-com:office:office"
xmlns:w="urn:schemas-microsoft-com:office:word"
xmlns:m="http://schemas.microsoft.com/office/2004/12/omml"
xmlns="http://www.w3.org/TR/REC-html40">
<meta name="Generator" content="Microsoft Word 14 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div link="blue" vlink="purple" lang="EN-US">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D">Hi
Robert,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Thank
you very much for sharing the information regarding
your efforts. Can you please share your idea of the
end to end flow? How do you suggest to bind Nova
and Neutron?</span></p>
</div>
</div>
</div>
</blockquote>
</span>
<div><br>
</div>
<div>The end to end flow is actually encompassed in the blueprints
in a nutshell. I will reiterate it in below. The binding between
Nova and Neutron occurs with the neutron v2 API that nova
invokes in order to provision the neutron services. The vif
driver is responsible for plugging in an instance onto the
networking setup that neutron has created on the host.</div>
<div><br>
</div>
<div>Normally, one will invoke "nova boot" api with the —nic
options to specify the nic with which the instance will be
connected to the network. It currently allows net-id, fixed ip
and/or port-id to be specified for the option. However, it
doesn't allow one to specify special networking requirements for
the instance. Thanks to the nova pci-passthrough work, one can
specify PCI passthrough device(s) in the nova flavor. But it
doesn't provide means to tie up these PCI devices in the case of
ethernet adpators with networking services. Therefore the idea
is actually simple as indicated by the blueprint titles, to
provide means to tie up SRIOV devices with neutron services. A
work flow would roughly look like this for 'nova boot':</div>
<div><br>
</div>
<div> -- Specifies networking requirements in the —nic
option. Specifically for SRIOV, allow the following to be
specified in addition to the existing required information:</div>
<div> . PCI alias</div>
<div> . direct pci-passthrough/macvtap</div>
<div> . port profileid that is compliant with
802.1Qbh</div>
<div> </div>
<div> The above information is optional. In the absence of
them, the existing behavior remains.</div>
<div><br>
</div>
<div> -- if special networking requirements exist, Nova api
creates PCI requests in the nova instance type for scheduling
purpose</div>
<div><br>
</div>
<div> -- Nova scheduler schedules the instance based on the
requested flavor plus the PCI requests that are created for
networking.</div>
<div><br>
</div>
<div> -- Nova compute invokes neutron services with PCI
passthrough information if any </div>
<div><br>
</div>
<div> -- Neutron performs its normal operations based on the
request, such as allocating a port, assigning ip addresses, etc.
Specific to SRIOV, it should validate the information such as
profileid, and stores them in its db. It's also possible to
associate a port profileid with a neutron network so that port
profileid becomes optional in the —nic option. Neutron returns
nova the port information, especially for PCI passthrough
related information in the port binding object. Currently, the
port binding object contains the following information:</div>
<div> binding:vif_type</div>
<div> binding:host_id</div>
<div> binding:profile</div>
<div> binding:capabilities</div>
</blockquote>
(openstack bonce stop me to sent to so many people at one time, so i
remove cc & to, hope every one can see this)<br>
<br>
i heard of some nic passthrough solution in summary， and you
metioned this， in high level of implement the NIC passthrough there
is: <br>
hardware VEB (Virtual ethernet Switches )<br>
the nic need external switch like 802.1qbg <br>
<br>
so question is: <br>
where is the diffrent type infomation?<br>
does 802.1qbg need know which port the PF connected to?
<blockquote
cite="mid:1705659276F6A540A34DBC19195799841027A7A4@xmb-rcd-x03.cisco.com"
type="cite">
<div><br>
</div>
<div> -- nova constructs the domain xml and plug in the
instance by calling the vif driver. The vif driver can build up
the interface xml based on the port binding information. </div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE"
style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0
0 0 5;">
<div xmlns:v="urn:schemas-microsoft-com:vml"
xmlns:o="urn:schemas-microsoft-com:office:office"
xmlns:w="urn:schemas-microsoft-com:office:word"
xmlns:m="http://schemas.microsoft.com/office/2004/12/omml"
xmlns="http://www.w3.org/TR/REC-html40">
<div link="blue" vlink="purple" lang="EN-US">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">The
blueprints you registered make sense. On Nova side,
there is a need to bind between requested virtual
network and PCI device/interface to be allocated as
vNIC.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">On the
Neutron side, there is a need to support networking
configuration of the vNIC. Neutron should be able to
identify the PCI device/macvtap interface in order
to apply configuration. I think it makes sense to
provide neutron integration via dedicated Modular
Layer 2 Mechanism Driver to allow PCI pass-through
vNIC support along with other networking
technologies.</span></p>
</div>
</div>
</div>
</blockquote>
</span>
<div><br>
</div>
<div>I haven't sorted through this yet. A neutron port could be
associated with a PCI device or not, which is a common feature,
IMHO. However, a ML2 driver may be needed specific to a
particular SRIOV technology. </div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE"
style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0
0 0 5;">
<div xmlns:v="urn:schemas-microsoft-com:vml"
xmlns:o="urn:schemas-microsoft-com:office:office"
xmlns:w="urn:schemas-microsoft-com:office:word"
xmlns:m="http://schemas.microsoft.com/office/2004/12/omml"
xmlns="http://www.w3.org/TR/REC-html40">
<div link="blue" vlink="purple" lang="EN-US">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">During
the Havana Release, we introduced Mellanox Neutron
plugin that enables networking via SRIOV
pass-through devices or macvtap interfaces.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">We want
to integrate our solution with PCI pass-through Nova
support. I will be glad to share more details if
you are interested.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
</div>
</div>
</div>
</blockquote>
</span>
<div><br>
</div>
<div>Good to know that you already have a SRIOV implementation. I
found out some information online about the mlnx plugin, but
need more time to get to know it better. And certainly I'm
interested in knowing its details.</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE"
style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0
0 0 5;">
<div xmlns:v="urn:schemas-microsoft-com:vml"
xmlns:o="urn:schemas-microsoft-com:office:office"
xmlns:w="urn:schemas-microsoft-com:office:word"
xmlns:m="http://schemas.microsoft.com/office/2004/12/omml"
xmlns="http://www.w3.org/TR/REC-html40">
<div link="blue" vlink="purple" lang="EN-US">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D">The PCI
pass-through networking support is planned to be
discussed during the summit:
</span><a moz-do-not-send="true"
href="http://summit.openstack.org/cfp/details/129">http://summit.openstack.org/cfp/details/129</a>.
<span style="color:#1F497D">I think it’s worth to
drill down into more detailed proposal and present
it during the summit, especially since it impacts
both nova and neutron projects.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
</div>
</div>
</div>
</blockquote>
</span>
<div>I agree. Maybe we can steal some time in that discussion.</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE"
style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0
0 0 5;">
<div xmlns:v="urn:schemas-microsoft-com:vml"
xmlns:o="urn:schemas-microsoft-com:office:office"
xmlns:w="urn:schemas-microsoft-com:office:word"
xmlns:m="http://schemas.microsoft.com/office/2004/12/omml"
xmlns="http://www.w3.org/TR/REC-html40">
<div link="blue" vlink="purple" lang="EN-US">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D">Would
you be interested in collaboration on this effort?
Would you be interested to exchange more emails or
set an IRC/WebEx meeting during this week before the
summit?<o:p></o:p></span></p>
</div>
</div>
</div>
</blockquote>
</span>
<div><br>
</div>
<div>Sure. If folks want to discuss it before the summit, we can
schedule a webex later this week. Or otherwise, we can continue
the discussion with email.</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE"
style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0
0 0 5;">
<div xmlns:v="urn:schemas-microsoft-com:vml"
xmlns:o="urn:schemas-microsoft-com:office:office"
xmlns:w="urn:schemas-microsoft-com:office:word"
xmlns:m="http://schemas.microsoft.com/office/2004/12/omml"
xmlns="http://www.w3.org/TR/REC-html40">
<div link="blue" vlink="purple" lang="EN-US">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Irena<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF
1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:
10pt; font-family: Tahoma, sans-serif; ">From:</span></b><span
style="font-size: 10pt; font-family: Tahoma,
sans-serif; "> Robert Li (baoli) [<a
moz-do-not-send="true"
href="mailto:baoli@cisco.com">mailto:baoli@cisco.com</a>]
<br>
<b>Sent:</b> Friday, October 25, 2013 11:16 PM<br>
<b>To:</b> <a moz-do-not-send="true"
href="mailto:prashant.upadhyaya@aricent.com">prashant.upadhyaya@aricent.com</a>;
Irena Berezovsky;
<a moz-do-not-send="true"
href="mailto:yunhong.jiang@intel.com">yunhong.jiang@intel.com</a>;
<a moz-do-not-send="true"
href="mailto:chris.friesen@windriver.com">
chris.friesen@windriver.com</a>; <a
moz-do-not-send="true"
href="mailto:yongli.he@intel.com">yongli.he@intel.com</a><br>
<b>Cc:</b> OpenStack Development Mailing List;
Brian Bowen (brbowen); Kyle Mestery (kmestery);
Sandhya Dasu (sadasu)<br>
<b>Subject:</b> Re: [openstack-dev] [nova]
[neutron] PCI pass-through network support<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:black">Hi Irena,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:black">This is
Robert Li from Cisco Systems. Recently, I was
tasked to investigate such support for Cisco's
systems that support VM-FEX, which is a SRIOV
technology supporting 802-1Qbh. I was able to
bring up nova instances with SRIOV interfaces, and
establish networking in between the instances that
employes the SRIOV interfaces. Certainly, this was
accomplished with hacking and some manual
intervention. Based on this experience and my
study with the two existing nova pci-passthrough
blueprints that have been implemented and
committed into Havana (<a moz-do-not-send="true"
href="https://blueprints.launchpad.net/nova/+spec/pci-passthrough-base"
id="yui_3_10_3_1_1382711548329_391"><span
style="font-size: 9pt; font-family: Arial,
sans-serif; color: rgb(0, 51, 170);
background-color: white; text-decoration:
none; ">https://blueprints.launchpad.net/nova/+spec/pci-passthrough-base</span></a></span><span
style="font-size: 9pt; font-family: Arial,
sans-serif; color: rgb(51, 51, 51);
background-color: white; "> and<br>
</span><span style="font-size: 9pt; font-family:
Arial, sans-serif; color: rgb(0, 51, 170);
background-color: white; "><a
moz-do-not-send="true"
href="https://blueprints.launchpad.net/nova/+spec/pci-passthrough-libvirt"><span
style="font-size:10.5pt">https://blueprints.launchpad.net/nova/+spec/pci-passthrough-libvirt</span></a>), </span><span
style="font-size:10.5pt;color:black"> I registered
a couple of blueprints (one on Nova side, the
other on the Neutron side):<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:black"><a
moz-do-not-send="true"
href="https://blueprints.launchpad.net/nova/+spec/pci-passthrough-sriov">https://blueprints.launchpad.net/nova/+spec/pci-passthrough-sriov</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:black"><a
moz-do-not-send="true"
href="https://blueprints.launchpad.net/neutron/+spec/pci-passthrough-sriov">https://blueprints.launchpad.net/neutron/+spec/pci-passthrough-sriov</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:black">in order to
address SRIOV support in openstack. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:black">Please take a
look at them and see if they make sense, and let
me know any comments and questions. We can also
discuss this in the summit, I suppose.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:black">I noticed
that there is another thread on this topic, so
copy those folks from that thread as well.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:black">thanks,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:black">Robert<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:black">On 10/16/13
4:32 PM, "Irena Berezovsky" <<a
moz-do-not-send="true"
href="mailto:irenab@mellanox.com">irenab@mellanox.com</a>>
wrote:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:black"><o:p> </o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #B5C4DF
4.5pt;padding:0in 0in 0in
4.0pt;margin-left:3.75pt;margin-right:0in"
id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE">
<div>
<div>
<p class="MsoNormal"><span style="color:black">Hi,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">As
one of the next steps for PCI pass-through I
would like to discuss is the support for PCI
pass-through vNIC.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">While
nova takes care of PCI pass-through device
resources management and VIF settings,
neutron should manage their networking
configuration.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">I
would like to register a</span><span
style="color:#1F497D"></span><span
style="color:black">summit proposal to discuss
the support for PCI pass-through networking.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">I
am not sure what would be the right topic to
discuss the PCI pass-through networking, since
it involve both nova and neutron. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">There
is already a session registered by Yongli on
nova topic to discuss the PCI pass-through
next steps.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">I
think PCI pass-through networking is quite a
big topic and it worth to have a separate
discussion.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Is
there any other people who are interested to
discuss it and share their thoughts and
experience?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Irena<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</span>
</blockquote>
<br>
</body>
</html>