<div dir="ltr"><div><div><div>Reiterating from the IRC mneeting, largely, so apologies.<br><br></div><div>Firstly, I disagree that <a href="https://wiki.openstack.org/wiki/PCI_passthrough_SRIOV_support">https://wiki.openstack.org/wiki/PCI_passthrough_SRIOV_support</a> is an accurate reflection of the current state. It's a very unilateral view, largely because the rest of us had been focussing on the google document that we've been using for weeks.<br>
</div><div><br>Secondly, I totally disagree with this approach. This assumes that description of the (cloud-internal, hardware) details of each compute node is best done with data stored centrally and driven by an API. I don't agree with either of these points.<br>
<br></div>Firstly, the best place to describe what's available on a compute node is in the configuration on the compute node. For instance, I describe which interfaces do what in Neutron on the compute node. This is because when you're provisioning nodes, that's the moment you know how you've attached it to the network and what hardware you've put in it and what you intend the hardware to be for - or conversely your deployment puppet or chef or whatever knows it, and Razor or MAAS has enumerated it, but the activities are equivalent. Storing it centrally distances the compute node from its descriptive information for no good purpose that I can see and adds the complexity of having to go make remote requests just to start up.<br>
<br></div>Secondly, even if you did store this centrally, it's not clear to me that an API is very useful. As far as I can see, the need for an API is really the need to manage PCI device flavors. If you want that to be API-managed, then the rest of a (rather complex) API cascades from that one choice. Most of the things that API lets you change (expressions describing PCI devices) are the sort of thing that you set once and only revisit when you start - for instance - deploying new hosts in a different way.<br>
<br>I at the parallel in Neutron provider networks. They're config driven, largely on the compute hosts. Agents know what ports on their machine (the hardware tie) are associated with provider networks, by provider network name. The controller takes 'neutron net-create ... --provider:network 'name'' and uses that to tie a virtual network to the provider network definition on each host. What we absolutely don't do is have a complex admin API that lets us say 'in host aggregate 4, provider network x (which I made earlier) is connected to eth6'.<br>
<br>-- <br></div>Ian.<br><div><div><br></div></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On 17 December 2013 03:12, yongli he <span dir="ltr"><<a href="mailto:yongli.he@intel.com" target="_blank">yongli.he@intel.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div>On 2013年12月16日 22:27, Robert Li (baoli)
wrote:<br>
</div>
<blockquote type="cite">
<div>Hi Yongli,</div>
<div><br>
</div>
<div>The IRC meeting we have for PCI-Passthrough is the forum for
discussion on SR-IOV support in openstack. I think the goal is
to come up with a plan on both the nova and neutron side in
support of the SR-IOV, and the current focus is on the nova
side. Since you've done a lot of work on it already, would you
like to lead tomorrow's discussion at UTC 1400? <br>
</div>
</blockquote>
<br>
Robert , you lead the meeting very well i enjoy you setup every for
us, keep going on it -:)<br>
<br>
I'd like to give you guy a summary of current state, let's discuss
it then. <br>
<a href="https://wiki.openstack.org/wiki/PCI_passthrough_SRIOV_support" target="_blank">https://wiki.openstack.org/wiki/PCI_passthrough_SRIOV_support</a><br>
<br>
<br>
1) fade out alias ( i think this ok for all)<br>
2) white list became pic-flavor ( i think this ok for all)<br>
3) address simply regular expression support: only * and a number
range is support [hex-hex]. ( i think this ok?)<br>
4) aggregate : now it's clear enough, and won't impact SRIOV. ( i
think this irrelevant to SRIOV now)<br>
<br>
<br>
5) SRIOV use case, if you suggest a use case, please given a full
example like this: [discuss: compare to other solution]<br>
<br>
<ul style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:start;font-style:normal;font-weight:normal;padding:0px;line-height:20px;color:rgb(51,51,51);text-transform:none;font-size:14px;white-space:normal;margin:0.3em 0px 0px 1.6em;font-family:'Arial Unicode MS',Arial,sans-serif;word-spacing:0px">
<li style="line-height:20px">create a pci flavor for the SRIOV</li>
</ul>
<pre style="padding:9.5px;font-family:Monaco,Menlo,Consolas,'Courier New',monospace;font-size:13px;color:rgb(51,51,51);border-top-left-radius:4px;border-top-right-radius:4px;border-bottom-right-radius:4px;border-bottom-left-radius:4px;display:block;margin:0px 0px 10px;line-height:20px;word-break:break-all;word-wrap:break-word;white-space:pre-wrap;background-color:rgb(245,245,245);border:1px solid rgba(0,0,0,0.14902);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px">
nova pci-flavor-create name 'vlan-SRIOV' description "xxxxx"
nova pci-flavor-update UUID set 'description'='xxxx' 'address'= '0000:01:*.7'
</pre>
<h5 style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:start;font-style:normal;font-weight:bold;line-height:20px;color:rgb(51,51,51);text-transform:none;font-size:14px;white-space:normal;margin:10px 0px;font-family:'Arial Unicode MS',Arial,sans-serif;word-spacing:0px">
<br>
<span></span></h5>
<h5 style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:start;font-style:normal;font-weight:bold;line-height:20px;color:rgb(51,51,51);text-transform:none;font-size:14px;white-space:normal;margin:10px 0px;font-family:'Arial Unicode MS',Arial,sans-serif;word-spacing:0px">
<span>Admin config SRIOV</span></h5>
<ul style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:start;font-style:normal;font-weight:normal;padding:0px;line-height:20px;color:rgb(51,51,51);text-transform:none;font-size:14px;white-space:normal;margin:0.3em 0px 0px 1.6em;font-family:'Arial Unicode MS',Arial,sans-serif;word-spacing:0px">
<li style="line-height:20px">create pci-flavor :</li>
</ul>
<pre style="padding:9.5px;font-family:Monaco,Menlo,Consolas,'Courier New',monospace;font-size:13px;color:rgb(51,51,51);border-top-left-radius:4px;border-top-right-radius:4px;border-bottom-right-radius:4px;border-bottom-left-radius:4px;display:block;margin:0px 0px 10px;line-height:20px;word-break:break-all;word-wrap:break-word;white-space:pre-wrap;background-color:rgb(245,245,245);border:1px solid rgba(0,0,0,0.14902);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px">
{"name": "privateNIC", "neutron-network-uuid": "uuid-1", ...}
{"name": "publicNIC", "neutron-network-uuid": "uuid-2", ...}
{"name": "smallGPU", "neutron-network-uuid": "", ...}
</pre>
<ul style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:start;font-style:normal;font-weight:normal;padding:0px;line-height:20px;color:rgb(51,51,51);text-transform:none;font-size:14px;white-space:normal;margin:0.3em 0px 0px 1.6em;font-family:'Arial Unicode MS',Arial,sans-serif;word-spacing:0px">
<li style="line-height:20px">set aggregate meta according the
flavors existed in the hosts</li>
</ul>
<p style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:start;font-style:normal;font-weight:normal;line-height:20px;color:rgb(51,51,51);text-transform:none;font-size:14px;white-space:normal;margin:0px 0px 10px;font-family:'Arial Unicode MS',Arial,sans-serif;word-spacing:0px">
flavor extra-specs, for a VM that gets two small GPUs and
VIFs attached from the above SRIOV NICs:</p>
<pre style="padding:9.5px;font-family:Monaco,Menlo,Consolas,'Courier New',monospace;font-size:13px;color:rgb(51,51,51);border-top-left-radius:4px;border-top-right-radius:4px;border-bottom-right-radius:4px;border-bottom-left-radius:4px;display:block;margin:0px 0px 10px;line-height:20px;word-break:break-all;word-wrap:break-word;white-space:pre-wrap;background-color:rgb(245,245,245);border:1px solid rgba(0,0,0,0.14902);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px">
nova aggregate-set-metadata pci-aware-group set 'pci-flavor'='smallGPU,oldGPU, privateNIC,privateNIC'
</pre>
<ul style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:start;font-style:normal;font-weight:normal;padding:0px;line-height:20px;color:rgb(51,51,51);text-transform:none;font-size:14px;white-space:normal;margin:0.3em 0px 0px 1.6em;font-family:'Arial Unicode MS',Arial,sans-serif;word-spacing:0px">
<li style="line-height:20px">create instance flavor for sriov</li>
</ul>
<pre style="padding:9.5px;font-family:Monaco,Menlo,Consolas,'Courier New',monospace;font-size:13px;color:rgb(51,51,51);border-top-left-radius:4px;border-top-right-radius:4px;border-bottom-right-radius:4px;border-bottom-left-radius:4px;display:block;margin:0px 0px 10px;line-height:20px;word-break:break-all;word-wrap:break-word;white-space:pre-wrap;background-color:rgb(245,245,245);border:1px solid rgba(0,0,0,0.14902);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px">
nova flavor-key 100 set 'pci-flavor='1:privateNIC; 1: publicNIC; 2:smallGPU,oldGPU'
</pre>
<ul style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:start;font-style:normal;font-weight:normal;padding:0px;line-height:20px;color:rgb(51,51,51);text-transform:none;font-size:14px;white-space:normal;margin:0.3em 0px 0px 1.6em;font-family:'Arial Unicode MS',Arial,sans-serif;word-spacing:0px">
<li style="line-height:20px">User just specifies a quantum port
as normal:</li>
</ul>
<pre style="padding:9.5px;font-family:Monaco,Menlo,Consolas,'Courier New',monospace;font-size:13px;color:rgb(51,51,51);border-top-left-radius:4px;border-top-right-radius:4px;border-bottom-right-radius:4px;border-bottom-left-radius:4px;display:block;margin:0px 0px 10px;line-height:20px;word-break:break-all;word-wrap:break-word;white-space:pre-wrap;background-color:rgb(245,245,245);border:1px solid rgba(0,0,0,0.14902);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px">
nova boot --flavor "sriov-plus-two-gpu" --image img --nic net-id=uuid-2 --nic net-id=uuid-1 vm-name</pre>
<br>
<br>
Yongli <br>
<br>
<br>
<blockquote type="cite">
<div><br>
</div>
<div>Thanks,</div>
<div>Robert</div>
<div><br>
</div>
<span>
<div>
<div>On 12/11/13 8:09 PM, "He, Yongli" <<a href="mailto:yongli.he@intel.com" target="_blank">yongli.he@intel.com</a>>
wrote:</div>
</div>
<div><br>
</div>
<blockquote style="BORDER-LEFT:#b5c4df 5 solid;PADDING:0 0 0 5;MARGIN:0 0 0 5">
<div>
<div link="blue" vlink="purple" lang="ZH-CN">
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1f497d"></span><span style="font-size:10.5pt;color:#1f497d" lang="EN-US">Hi, all<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1f497d" lang="EN-US">Please continue to foucs on the
blueprint, it change after reviewing. And for this
point:<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1f497d" lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><br>
>5. flavor style for sriov: i just list the
flavor style in the design but for the style
<br>
> --nic <br>
> --pci-flavor PowerfullNIC:1<br>
> still possible to work, so what's the real
impact to sriov from the flavor design?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">>As
you can see from the log, Irena has some strong
opinions on this, and I tend to agree with her. The
problem we need to solve is this: we need a means to
associate a nic (or port) with a PCI device that is
allocated out of a PCI >flavor or a PCI group. We
think that we presented a complete solution in our
google doc.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">It’s
not so clear, could you please list the key point
here. Btw, the blue print I sent Monday had changed
for this, please check.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Yongli
he<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1f497d" lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1f497d" lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1f497d" lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1f497d" lang="EN-US"><u></u> <u></u></span></p>
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">From:</span></b><span style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US"> Robert Li (baoli) [<a href="mailto:baoli@cisco.com" target="_blank">mailto:baoli@cisco.com</a>]
<br>
<b>Sent:</b> Wednesday, December 11, 2013 10:18
PM<br>
<b>To:</b> He, Yongli; Sandhya Dasu (sadasu);
OpenStack Development Mailing List (not for
usage questions); Jiang, Yunhong; Irena
Berezovsky;
<a href="mailto:prashant.upadhyaya@aricent.com" target="_blank">prashant.upadhyaya@aricent.com</a>;
<a href="mailto:chris.friesen@windriver.com" target="_blank">chris.friesen@windriver.com</a>;
Itzik Brown;
<a href="mailto:john@johngarbutt.com" target="_blank">john@johngarbutt.com</a><br>
<b>Subject:</b> Re: [openstack-dev] [nova]
[neutron] PCI pass-through network support<u></u><u></u></span></p>
</div>
</div>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Hi
Yongli,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Thank
you very much for sharing the Wiki with us on
Monday so that we have a better understanding on
your ideas and thoughts. Please see embedded
comments.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">--Robert<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">On 12/10/13 8:35 PM, "yongli he"
<<a href="mailto:yongli.he@intel.com" target="_blank">yongli.he@intel.com</a>>
wrote:<u></u><u></u></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<blockquote style="border:none;border-left:solid #b5c4df 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-right:0cm">
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">On 2013</span><span style="font-size:10.5pt;font-family:PMingLiU,serif" lang="ZH-TW">年</span><span style="font-size:10.5pt" lang="EN-US">12</span><span style="font-size:10.5pt;font-family:PMingLiU,serif" lang="ZH-TW">月</span><span style="font-size:10.5pt" lang="EN-US">10</span><span style="font-size:10.5pt;font-family:PMingLiU,serif" lang="ZH-TW">日</span><span style="font-size:10.5pt" lang="EN-US"> 22:41, Sandhya Dasu (sadasu)
wrote:<u></u><u></u></span></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Hi,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> I am trying to resurrect
this email thread since discussions have
split between several threads and is
becoming hard to keep track.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">An update:<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">New PCI Passthrough meeting
time: Tuesdays UTC 1400.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">New PCI flavor proposal from
Nova:<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><a href="https://wiki.openstack.org/wiki/PCI_configration_Database_and_API#Take_advantage_of_host_aggregate_.28T.B.D.29" target="_blank">https://wiki.openstack.org/wiki/PCI_configration_Database_and_API#Take_advantage_of_host_aggregate_.28T.B.D.29</a><u></u><u></u></span></p>
</div>
</blockquote>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Hi, all<br>
sorry for miss the meeting, i was seeking
John at that time. from the log i saw some
concern about new design, i list them there
and try to clarify it per my opinion:<br>
<br>
1. configuration going to deprecated: this
might impact SRIOV. if possible, please list
what kind of impact make to you.
<u></u><u></u></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Regarding
the nova API pci-flavor-update, we had a
face-to-face discussion over use of a nova API to
provision/define/configure PCI passthrough list
during the ice-house summit. I kind of like the
idea initially. As you can see from the meeting
log, however, I later thought that in a
distributed system, using a centralized API to
define resources per compute node, which could
come and go any time, doesn't seem to provide any
significant benefit. This is the reason that I
didn't mention it in our google doc <a href="https://docs.google.com/document/d/1EMwDg9J8zOxzvTnQJ9HwZdiotaVstFWKIuKrPse6JOs/edit#" target="_blank">https://docs.google.com/document/d/1EMwDg9J8zOxzvTnQJ9HwZdiotaVstFWKIuKrPse6JOs/edit#</a><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">If
you agree that pci-flavor and pci-group is kind of
the same thing, then we agree with you that the
pci-flavor-create API is needed. Since pci-flavor
or pci-group is global, then such an API can be
used for resource registration/validation on nova
server. In addition, it can be used to facilitate
the display of PCI devices per node, per group, or
in the entire cloud, etc.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<blockquote style="border:none;border-left:solid #b5c4df 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><br>
<br>
2. <baoli>So the API seems to be
combining the whitelist + pci-group<br>
yeah, it's actually almost same thing,
'flavor' 'pci-group' or 'group'. the real
different is this flavor going to deprecated
the alias, and combine tight to aggregate or
flavor.
<u></u><u></u></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Well,
with pci-group, we recommended to deprecate the
PCI alias because we think it is redundant. <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">We
think that specification of PCI requirement in the
flavor's extra spec is still needed as it's a
generic means to allocate PCI devices. In
addition, it can be used as properties in the host
aggregate as well. <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<blockquote style="border:none;border-left:solid #b5c4df 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><br>
<br>
3. feature: <br>
this design is not to say the feature is
not work, but changed. if auto discovery
feature is possible, we got 'feature' form the
device, then use the feature to define the
pci-flavor. it's also possible create default
pci-flavor for this. so the feature concept
will be impact, my feeling, we should given a
separated bp for feature, and not in this
round change, so here we only thing is keep
the feature is possible.
<u></u><u></u></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">I
think that it's ok to have separate BPs. But we
think that auto discovery is an essential part of
the design, and therefore it should be implemented
with more helping hands. <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<blockquote style="border:none;border-left:solid #b5c4df 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><br>
<br>
4. address regular expression: i'm fine with
the wild-match style.<u></u><u></u></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Sounds
good. One side node is that I noticed that the
driver for intel 82576 cards has a strange slot
assignment scheme. So the final definition of it
may need to accommodate that as well.<u></u><u></u></span></p>
</div>
<blockquote style="border:none;border-left:solid #b5c4df 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><br>
<br>
5. flavor style for sriov: i just list the
flavor style in the design but for the style
<br>
--nic <br>
--pci-flavor
PowerfullNIC:1<br>
still possible to work, so what's the real
impact to sriov from the flavor design?<u></u><u></u></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">As
you can see from the log, Irena has some strong
opinions on this, and I tend to agree with her.
The problem we need to solve is this: we need a
means to associate a nic (or port) with a PCI
device that is allocated out of a PCI flavor or a
PCI group. We think that we presented a complete
solution in our google doc.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">At
this point, I really believe that we should
combine our efforts and ideas. As far as how many
BPs are needed, it should be a trivial matter
after we have agreed on a complete solution.<u></u><u></u></span></p>
</div>
<blockquote style="border:none;border-left:solid #b5c4df 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><br>
<br>
Yongli He<br>
<br>
<br>
<br>
<u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Thanks,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Sandhya<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style lang="EN-US">From: </span></b><span style lang="EN-US">Sandhya
Dasu <<a href="mailto:sadasu@cisco.com" target="_blank">sadasu@cisco.com</a>><br>
<b>Reply-To: </b>"OpenStack Development
Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>><br>
<b>Date: </b>Thursday, November 7, 2013
9:44 PM<br>
<b>To: </b>"OpenStack Development Mailing
List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>>,
"Jiang, Yunhong" <<a href="mailto:yunhong.jiang@intel.com" target="_blank">yunhong.jiang@intel.com</a>>,
"Robert Li (baoli)" <<a href="mailto:baoli@cisco.com" target="_blank">baoli@cisco.com</a>>,
Irena Berezovsky <<a href="mailto:irenab@mellanox.com" target="_blank">irenab@mellanox.com</a>>,
"<a href="mailto:prashant.upadhyaya@aricent.com" target="_blank">prashant.upadhyaya@aricent.com</a>"
<<a href="mailto:prashant.upadhyaya@aricent.com" target="_blank">prashant.upadhyaya@aricent.com</a>>,
"<a href="mailto:chris.friesen@windriver.com" target="_blank">chris.friesen@windriver.com</a>"
<<a href="mailto:chris.friesen@windriver.com" target="_blank">chris.friesen@windriver.com</a>>,
"He, Yongli" <<a href="mailto:yongli.he@intel.com" target="_blank">yongli.he@intel.com</a>>,
Itzik Brown <<a href="mailto:ItzikB@mellanox.com" target="_blank">ItzikB@mellanox.com</a>><br>
<b>Subject: </b>Re: [openstack-dev] [nova]
[neutron] PCI pass-through network support<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Hi,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> The discussions during
the summit were very productive. Now, we
are ready to setup our IRC meeting.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Here are some slots that
look like they might work for us.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">1. Wed 2 – 3 pm UTC.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">2. Thursday 12 – 1 pm UTC.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">3. Thursday 7 – 8pm UTC.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Please vote.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Thanks,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Sandhya<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style lang="EN-US">From:
</span></b><span style lang="EN-US">Sandhya Dasu <<a href="mailto:sadasu@cisco.com" target="_blank">sadasu@cisco.com</a>><br>
<b>Reply-To: </b>"OpenStack Development
Mailing List (not for usage questions)"
<<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>><br>
<b>Date: </b>Tuesday, November 5, 2013
12:03 PM<br>
<b>To: </b>"OpenStack Development
Mailing List (not for usage questions)"
<<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>>,
"Jiang, Yunhong" <<a href="mailto:yunhong.jiang@intel.com" target="_blank">yunhong.jiang@intel.com</a>>,
"Robert Li (baoli)" <<a href="mailto:baoli@cisco.com" target="_blank">baoli@cisco.com</a>>,
Irena Berezovsky <<a href="mailto:irenab@mellanox.com" target="_blank">irenab@mellanox.com</a>>,
"<a href="mailto:prashant.upadhyaya@aricent.com" target="_blank">prashant.upadhyaya@aricent.com</a>"
<<a href="mailto:prashant.upadhyaya@aricent.com" target="_blank">prashant.upadhyaya@aricent.com</a>>,
"<a href="mailto:chris.friesen@windriver.com" target="_blank">chris.friesen@windriver.com</a>"
<<a href="mailto:chris.friesen@windriver.com" target="_blank">chris.friesen@windriver.com</a>>,
"He, Yongli" <<a href="mailto:yongli.he@intel.com" target="_blank">yongli.he@intel.com</a>>,
Itzik Brown <<a href="mailto:ItzikB@mellanox.com" target="_blank">ItzikB@mellanox.com</a>><br>
<b>Subject: </b>Re: [openstack-dev]
[nova] [neutron] PCI pass-through
network support<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Just to clarify, the
discussion is planned for 10 AM
Wednesday morning at the developer's
lounge.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Thanks,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Sandhya<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style lang="EN-US">From:
</span></b><span style lang="EN-US">Sandhya Dasu <<a href="mailto:sadasu@cisco.com" target="_blank">sadasu@cisco.com</a>><br>
<b>Reply-To: </b>"OpenStack
Development Mailing List (not for
usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>><br>
<b>Date: </b>Tuesday, November 5,
2013 11:38 AM<br>
<b>To: </b>"OpenStack Development
Mailing List (not for usage
questions)" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>>,
"Jiang, Yunhong" <<a href="mailto:yunhong.jiang@intel.com" target="_blank">yunhong.jiang@intel.com</a>>,
"Robert Li (baoli)" <<a href="mailto:baoli@cisco.com" target="_blank">baoli@cisco.com</a>>,
Irena Berezovsky <<a href="mailto:irenab@mellanox.com" target="_blank">irenab@mellanox.com</a>>,
"<a href="mailto:prashant.upadhyaya@aricent.com" target="_blank">prashant.upadhyaya@aricent.com</a>"
<<a href="mailto:prashant.upadhyaya@aricent.com" target="_blank">prashant.upadhyaya@aricent.com</a>>,
"<a href="mailto:chris.friesen@windriver.com" target="_blank">chris.friesen@windriver.com</a>"
<<a href="mailto:chris.friesen@windriver.com" target="_blank">chris.friesen@windriver.com</a>>,
"He, Yongli" <<a href="mailto:yongli.he@intel.com" target="_blank">yongli.he@intel.com</a>>,
Itzik Brown <<a href="mailto:ItzikB@mellanox.com" target="_blank">ItzikB@mellanox.com</a>><br>
<b>Subject: </b>Re: [openstack-dev]
[nova] [neutron] PCI pass-through
network support<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><b><span style lang="EN-US">Hi,</span></b><span style="font-size:10.5pt" lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><b><span style lang="EN-US"> We are
planning to have a discussion
at the developer's lounge
tomorrow morning at 10:00 am.
Please feel free to drop by if
you are interested.</span></b><span style="font-size:10.5pt" lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><b><span style lang="EN-US">Thanks,</span></b><span style="font-size:10.5pt" lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><b><span style lang="EN-US">Sandhya</span></b><span style="font-size:10.5pt" lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><b><span style lang="EN-US">From: </span></b><span style lang="EN-US"><Jiang>,
Yunhong <</span><span style="font-size:10.5pt" lang="EN-US"><a href="mailto:yunhong.jiang@intel.com" target="_blank"><span style="font-size:11.0pt">yunhong.jiang@intel.com</span></a></span><span style lang="EN-US">></span><span style="font-size:10.5pt" lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style lang="EN-US">Date: </span></b><span style lang="EN-US">Thursday,
October 31, 2013 6:21 PM<br>
<b>To: </b>"Robert Li
(baoli)" <<a href="mailto:baoli@cisco.com" target="_blank">baoli@cisco.com</a>>,
Irena Berezovsky <<a href="mailto:irenab@mellanox.com" target="_blank">irenab@mellanox.com</a>>,
"<a href="mailto:prashant.upadhyaya@aricent.com" target="_blank">prashant.upadhyaya@aricent.com</a>"
<<a href="mailto:prashant.upadhyaya@aricent.com" target="_blank">prashant.upadhyaya@aricent.com</a>>,
"<a href="mailto:chris.friesen@windriver.com" target="_blank">chris.friesen@windriver.com</a>"
<<a href="mailto:chris.friesen@windriver.com" target="_blank">chris.friesen@windriver.com</a>>,
"He, Yongli" <<a href="mailto:yongli.he@intel.com" target="_blank">yongli.he@intel.com</a>>,
Itzik Brown <<a href="mailto:ItzikB@mellanox.com" target="_blank">ItzikB@mellanox.com</a>><br>
<b>Cc: </b>OpenStack
Development Mailing List
<<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>>,
"Brian Bowen (brbowen)" <<a href="mailto:brbowen@cisco.com" target="_blank">brbowen@cisco.com</a>>,
"Kyle Mestery (kmestery)"
<<a href="mailto:kmestery@cisco.com" target="_blank">kmestery@cisco.com</a>>,
Sandhya Dasu <<a href="mailto:sadasu@cisco.com" target="_blank">sadasu@cisco.com</a>><br>
<b>Subject: </b>RE:
[openstack-dev] [nova]
[neutron] PCI pass-through
network support<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1f497d" lang="EN-US">Robert, I
think your change request
for pci alias should be
covered by the extra infor
enhancement.
</span><span style lang="EN-US"><a href="https://blueprints.launchpad.net/nova/+spec/pci-extra-info" target="_blank">https://blueprints.launchpad.net/nova/+spec/pci-extra-info</a>
and Yongli is working on
it.<u></u><u></u></span></p>
<p class="MsoNormal"><span style lang="EN-US"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style lang="EN-US">I’m not sure
how the port profile is
passed to the connected
switch, is it a Cisco
VMEFX specific method or
libvirt method? Sorry I’m
not well on network side.<u></u><u></u></span></p>
<p class="MsoNormal"><span style lang="EN-US"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style lang="EN-US">--jyh<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1f497d" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">From:</span></b><span style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US"> Robert
Li (baoli) [<a href="mailto:baoli@cisco.com" target="_blank">mailto:baoli@cisco.com</a>]
<br>
<b>Sent:</b>
Wednesday, October
30, 2013 10:13 AM<br>
<b>To:</b> Irena
Berezovsky; Jiang,
Yunhong; <a href="mailto:prashant.upadhyaya@aricent.com" target="_blank">
prashant.upadhyaya@aricent.com</a>; <a href="mailto:chris.friesen@windriver.com" target="_blank">
chris.friesen@windriver.com</a>; He, Yongli; Itzik Brown<br>
<b>Cc:</b> OpenStack
Development Mailing
List; Brian Bowen
(brbowen); Kyle
Mestery (kmestery);
Sandhya Dasu
(sadasu)<br>
<b>Subject:</b> Re:
[openstack-dev]
[nova] [neutron] PCI
pass-through network
support</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
</div>
<p class="MsoNormal"><span style lang="EN-US"> <u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Hi,</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Regarding physical
network mapping, This
is what I thought. </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">consider the following
scenarios:</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> 1. a compute node
with SRIOV only
interfaces attached to
a physical network.
the node is connected
to one upstream
switch </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> 2. a compute node
with both SRIOV
interfaces and
non-SRIOV interfaces
attached to a physical
network. the node is
connected to one
upstream switch</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> 3. in addition to
case 1 &2, a
compute node may have
multiple vNICs that
are connected to
different upstream
switches. </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">CASE 1:</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> -- the mapping from a
virtual network (in
terms of neutron) to a
physical network is
actually done by
binding a port profile
to a neutron port.
With cisco's VM-FEX, a
port profile is
associated with one or
multiple vlans. Once
the neutron port is
bound with this
port-profile in the
upstream switch, it's
effectively plugged
into the physical
network. </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> -- since the compute
node is connected to
one upstream switch,
the existing nova PCI
alias will be
sufficient. For
example, one can boot
a Nova instance that
is attached to a SRIOV
port with the
following command:</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> nova boot
—flavor m1.large
—image
<image-id> --nic
net-id=<net>,pci-alias=<alias>,sriov=<direct|macvtap>,port-profile=<profile></span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> the net-id will be
useful in terms of
allocating IP address,
enable dhcp, etc that
is associated with the
network.</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">-- the pci-alias
specified in the nova
boot command is used
to create a PCI
request for scheduling
purpose. a PCI device
is bound to a neutron
port during the
instance build time in
the case of nova boot.
Before invoking the
neutron API to create
a port, an allocated
PCI device out of a
PCI alias will be
located from the PCI
device list object.
This device info among
other information will
be sent to neutron to
create the port. </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">CASE 2:</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">-- Assume that OVS is
used for the non-SRIOV
interfaces. An example
of configuration with
ovs plugin would look
like:</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">
bridge_mappings =
physnet1:br-vmfex</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">
network_vlan_ranges
= physnet1:15:17</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">
tenant_network_type
= vlan</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> When a neutron
network is created, a
vlan is either
allocated or specified
in the neutron
net-create command.
Attaching a physical
interface to the
bridge (in the above
example br-vmfex) is
an administrative
task. </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">-- to create a Nova
instance with
non-SRIOV port:</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> nova boot
—flavor m1.large
—image
<image-id> --nic
net-id=<net></span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">-- to create a Nova
instance with SRIOV
port:</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> nova boot
—flavor m1.large
—image
<image-id> --nic
net-id=<net>,pci-alias=<alias>,sriov=<direct|macvtap>,port-profile=<profile></span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> it's essentially
the same as in the
first case. But since
the net-id is already
associated with a
vlan, the vlan
associated with the
port-profile must be
identical to that
vlan. This has to be
enforced by neutron.</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> again, since the
node is connected to
one upstream switch,
the existing nova PCI
alias should be
sufficient. </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">CASE 3:</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">-- A compute node
might be connected to
multiple upstream
switches, with each
being a separate
network. This means
SRIOV PFs/VFs are
already implicitly
associated with
physical networks. In
the none-SRIOV case, a
physical interface is
associated with a
physical network by
plugging it into that
network, and attaching
this interface to the
ovs bridge that
represents this
physical network on
the compute node. In
the SRIOV case, we
need a way to group
the SRIOV VFs that
belong to the same
physical networks. The
existing nova PCI
alias is to facilitate
PCI device allocation
by associating
<product_id,
vendor_id> with an
alias name. This will
no longer be
sufficient. But it can
be enhanced to achieve
our goal. For example,
the PCI device domain,
bus (if their mapping
to vNIC is fixed
across boot) may be
added into the alias,
and the alias name
should be
corresponding to a
list of tuples. </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Another consideration
is that a VF or PF
might be used on the
host for other
purposes. For example,
it's possible for a
neutron DHCP server to
be bound with a VF.
Therefore, there needs
a method to exclude
some VFs from a group.
One way is to
associate an exclude
list with an alias.</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">The enhanced PCI alias
can be used to support
features other than
neutron as well.
Essentially, a PCI
alias can be defined
as a group of PCI
devices associated
with a feature. I'd
think that this should
be addressed with a
separate blueprint.</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Thanks,</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Robert</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">On 10/30/13 12:59 AM,
"Irena Berezovsky"
<<a href="mailto:irenab@mellanox.com" target="_blank">irenab@mellanox.com</a>> wrote:</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<blockquote style="border:none;border-left:solid #b5c4df 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US">Hi,</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US">Please see my answers inline</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">From:</span></b><span style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">
Jiang, Yunhong
[<a href="mailto:yunhong.jiang@intel.com" target="_blank">mailto:yunhong.jiang@intel.com</a>]
<br>
<b>Sent:</b>
Tuesday,
October 29,
2013 10:17 PM<br>
<b>To:</b>
Irena
Berezovsky;
Robert Li
(baoli); <a href="mailto:prashant.upadhyaya@aricent.com" target="_blank">
prashant.upadhyaya@aricent.com</a>; <a href="mailto:chris.friesen@windriver.com" target="_blank">
chris.friesen@windriver.com</a>; He, Yongli; Itzik Brown<br>
<b>Cc:</b>
OpenStack
Development
Mailing List;
Brian Bowen
(brbowen);
Kyle Mestery
(kmestery);
Sandhya Dasu
(sadasu)<br>
<b>Subject:</b>
RE:
[openstack-dev]
[nova]
[neutron] PCI
pass-through
network
support</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
</div>
<p class="MsoNormal"><span style lang="EN-US"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1f497d" lang="EN-US">Your explanation of
the virtual
network and
physical network
is quite clear and
should work well.
We need change
nova code to
achieve it,
including get the
physical network
for the virtual
network, passing
the physical
network
requirement to the
filter properties
etc.
</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><b><i><span style="font-size:10.5pt;color:#1f497d" lang="EN-US">[IrenaB]
</span></i></b><span style="font-size:10.5pt;color:#1f497d" lang="EN-US"> The physical
network is already
available to nova
at
networking/nova/api
at as virtual
network attribute,
it then passed to
the VIF driver. We
will push soon the
fix to:</span><span style lang="EN-US"><a href="https://bugs.launchpad.net/nova/+bug/1239606" target="_blank">https://bugs.launchpad.net/nova/+bug/1239606</a></span><span style="font-size:10.5pt;color:#1f497d" lang="EN-US"> ; which will
provide general
support for
getting this
information.</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1f497d" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1f497d" lang="EN-US">For your port
method, so you
mean we are sure
to passing network
id to ‘nova boot’
and nova will
create the port
during VM boot, am
I right? Also,
how can nova knows
that it need
allocate the PCI
device for the
port? I’d suppose
that in SR-IOV NIC
environment, user
don’t need specify
the PCI
requirement.
Instead, the PCI
requirement should
come from the
network
configuration and
image property. Or
you think user
still need passing
flavor with pci
request?</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><b><i><span style="color:#1f497d" lang="EN-US">[IrenaB] There are two way to apply
port method.
One is to pass
network id on
nova boot and
use default
type as chosen
in the neutron
config file
for vnic type.
Other way is
to define port
with required
vnic type and
other
properties if
applicable,
and run ‘nova
boot’ with
port id
argument.
Going forward
with nova
support for
PCI devices
awareness, we
do need a way
impact
scheduler
choice to land
VM on suitable
Host with
available PC
device that
has the
required
connectivity.</span></i></b><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1f497d" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1f497d" lang="EN-US">--jyh</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1f497d" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1f497d" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">From:</span></b><span style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">
Irena
Berezovsky [<a href="mailto:irenab@mellanox.com" target="_blank">mailto:irenab@mellanox.com</a>]
<br>
<b>Sent:</b>
Tuesday,
October 29,
2013 3:17 AM<br>
<b>To:</b>
Jiang,
Yunhong;
Robert Li
(baoli); <a href="mailto:prashant.upadhyaya@aricent.com" target="_blank">
prashant.upadhyaya@aricent.com</a>; <a href="mailto:chris.friesen@windriver.com" target="_blank">
chris.friesen@windriver.com</a>; He, Yongli; Itzik Brown<br>
<b>Cc:</b>
OpenStack
Development
Mailing List;
Brian Bowen
(brbowen);
Kyle Mestery
(kmestery);
Sandhya Dasu
(sadasu)<br>
<b>Subject:</b>
RE:
[openstack-dev]
[nova]
[neutron] PCI
pass-through
network
support</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
</div>
<p class="MsoNormal"><span style lang="EN-US"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US">Hi Jiang, Robert,</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US">IRC meeting option works for me.
</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US">If I understand your question below,
you are looking
for a way to tie
up between
requested
virtual
network(s) and
requested PCI
device(s). The
way we did it in
our solution is
to map a
provider:physical_network
to an interface
that represents
the Physical
Function. Every
virtual network
is bound to the
provider:physical_network,
so the PCI
device should be
allocated based
on this mapping.
We can map a
PCI alias to the
provider:physical_network.</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US">Another topic to discuss is where the
mapping between
neutron port and
PCI device
should be
managed. One way
to solve it, is
to propagate the
allocated PCI
device details
to neutron on
port creation.
</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US">In case there is no qbg/qbh support,
VF networking
configuration
should be
applied locally
on the Host.</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US">The question is when and how to apply
networking
configuration on
the PCI device?
</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US">We see the following options:</span><span style lang="EN-US"><u></u><u></u></span></p>
<p style="margin-left:38.7pt">
<span style lang="EN-US"><span>‧<span>
</span></span></span><span style="color:#1f497d" lang="EN-US">it can be
done on port
creation. </span><span style lang="EN-US"><u></u><u></u></span></p>
<p style="margin-left:38.7pt">
<span style lang="EN-US"><span>‧<span>
</span></span></span><span style="color:#1f497d" lang="EN-US">It can be
done when nova
VIF driver is
called for vNIC
plugging. This
will require to
have all
networking
configuration
available to the
VIF driver or
send request to
the neutron
server to obtain
it. </span><span style lang="EN-US"><u></u><u></u></span></p>
<p style="margin-left:38.7pt">
<span style lang="EN-US"><span>‧<span>
</span></span></span><span style="color:#1f497d" lang="EN-US">It can be
done by having
a dedicated L2
neutron agent on
each Host that
scans for
allocated PCI
devices and
then retrieves
networking
configuration
from the server
and configures
the device. The
agent will be
also responsible
for managing
update requests
coming from the
neutron server.
</span><span style lang="EN-US"><u></u><u></u></span></p>
<p style="margin-left:38.7pt"><span style="color:#1f497d" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US">For macvtap vNIC type assignment, the
networking
configuration
can be applied
by a dedicated
L2 neutron
agent.
</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US">BR,</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US">Irena</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">From:</span></b><span style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">
Jiang, Yunhong
[<a href="mailto:yunhong.jiang@intel.com" target="_blank">mailto:yunhong.jiang@intel.com</a>]
<br>
<b>Sent:</b>
Tuesday,
October 29,
2013 9:04 AM</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US"><br>
<b>To:</b>
Robert Li
(baoli); Irena
Berezovsky; <a href="mailto:prashant.upadhyaya@aricent.com" target="_blank">
prashant.upadhyaya@aricent.com</a>; <a href="mailto:chris.friesen@windriver.com" target="_blank">
chris.friesen@windriver.com</a>; He, Yongli; Itzik Brown<br>
<b>Cc:</b>
OpenStack
Development
Mailing List;
Brian Bowen
(brbowen);
Kyle Mestery
(kmestery);
Sandhya Dasu
(sadasu)<br>
<b>Subject:</b>
RE:
[openstack-dev]
[nova]
[neutron] PCI
pass-through
network
support</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
</div>
<p class="MsoNormal"><span style lang="EN-US"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1f497d" lang="EN-US">Robert, is it
possible to have
a IRC meeting?
I’d prefer to
IRC meeting
because it’s
more openstack
style and also
can keep the
minutes clearly.
</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1f497d" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1f497d" lang="EN-US">To your flow, can
you give more
detailed
example. For
example, I can
consider user
specify the
instance with
–nic option
specify a
network id, and
then how nova
device the
requirement to
the PCI device?
I assume the
network id
should define
the switches
that the device
can connect to ,
but how is that
information
translated to
the PCI property
requirement?
Will this
translation
happen before
the nova
scheduler make
host decision?</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1f497d" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1f497d" lang="EN-US">Thanks</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1f497d" lang="EN-US">--jyh</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1f497d" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">From:</span></b><span style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">
Robert Li
(baoli) [<a href="mailto:baoli@cisco.com" target="_blank">mailto:baoli@cisco.com</a>]
<br>
<b>Sent:</b>
Monday,
October 28,
2013 12:22 PM<br>
<b>To:</b>
Irena
Berezovsky; <a href="mailto:prashant.upadhyaya@aricent.com" target="_blank">prashant.upadhyaya@aricent.com</a>;
Jiang,
Yunhong;
<a href="mailto:chris.friesen@windriver.com" target="_blank">chris.friesen@windriver.com</a>;
He, Yongli;
Itzik Brown<br>
<b>Cc:</b>
OpenStack
Development
Mailing List;
Brian Bowen
(brbowen);
Kyle Mestery
(kmestery);
Sandhya Dasu
(sadasu)<br>
<b>Subject:</b>
Re:
[openstack-dev]
[nova]
[neutron] PCI
pass-through
network
support</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
</div>
<p class="MsoNormal"><span style lang="EN-US"> <u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Hi Irena,</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Thank you very much
for your
comments. See
inline. </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">--Robert</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">On 10/27/13 3:48 AM,
"Irena
Berezovsky"
<<a href="mailto:irenab@mellanox.com" target="_blank">irenab@mellanox.com</a>> wrote:</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<blockquote style="border:none;border-left:solid #b5c4df 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US">Hi Robert,</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US">Thank you very much for sharing the
information
regarding your
efforts. Can
you please
share your
idea of the
end to end
flow? How do
you suggest
to bind Nova
and Neutron?</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">The end to end flow is
actually
encompassed in
the blueprints
in a nutshell.
I will
reiterate it
in below. The
binding
between Nova
and Neutron
occurs with
the neutron v2
API that nova
invokes in
order to
provision the
neutron
services. The
vif driver is
responsible
for plugging
in an instance
onto the
networking
setup that
neutron has
created on the
host.</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Normally, one will
invoke "nova
boot" api with
the —nic
options to
specify the
nic with which
the instance
will be
connected to
the network.
It currently
allows net-id,
fixed ip
and/or port-id
to be
specified for
the option.
However, it
doesn't allow
one to specify
special
networking
requirements
for the
instance.
Thanks to the
nova
pci-passthrough
work, one can
specify PCI
passthrough
device(s) in
the nova
flavor. But it
doesn't
provide means
to tie up
these PCI
devices in the
case of
ethernet
adpators with
networking
services.
Therefore the
idea is
actually
simple as
indicated by
the blueprint
titles, to
provide means
to tie up
SRIOV devices
with neutron
services. A
work flow
would roughly
look like this
for 'nova
boot':</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> -- Specifies
networking
requirements
in the —nic
option.
Specifically
for SRIOV,
allow the
following to
be specified
in addition to
the existing
required
information:</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> . PCI
alias</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> .
direct
pci-passthrough/macvtap</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> . port
profileid that
is compliant
with 802.1Qbh</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> The above
information is
optional. In
the absence of
them, the
existing
behavior
remains.</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> -- if special
networking
requirements
exist, Nova
api creates
PCI requests
in the nova
instance type
for scheduling
purpose</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> -- Nova scheduler
schedules the
instance based
on the
requested
flavor plus
the PCI
requests that
are created
for
networking.</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> -- Nova compute
invokes
neutron
services with
PCI
passthrough
information if
any </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> -- Neutron
performs its
normal
operations
based on the
request, such
as allocating
a port,
assigning ip
addresses,
etc. Specific
to SRIOV, it
should
validate the
information
such as
profileid, and
stores them in
its db. It's
also possible
to associate a
port profileid
with a neutron
network so
that port
profileid
becomes
optional in
the —nic
option.
Neutron
returns nova
the port
information,
especially for
PCI
passthrough
related
information in
the port
binding
object.
Currently, the
port binding
object
contains the
following
information:</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">
binding:vif_type</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">
binding:host_id</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">
binding:profile</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">
binding:capabilities</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> -- nova constructs
the domain xml
and plug in
the instance
by calling the
vif driver.
The vif driver
can build up
the interface
xml based on
the port
binding
information. </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<blockquote style="border:none;border-left:solid #b5c4df 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US">The blueprints you registered make
sense. On Nova
side, there is
a need to bind
between
requested
virtual
network and
PCI
device/interface
to be
allocated as
vNIC.</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US">On the Neutron side, there is a need
to support
networking
configuration
of the vNIC.
Neutron should
be able to
identify the
PCI
device/macvtap
interface in
order to apply
configuration.
I think it
makes sense to
provide
neutron
integration
via dedicated
Modular Layer
2 Mechanism
Driver to
allow PCI
pass-through
vNIC support
along with
other
networking
technologies.</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">I haven't sorted
through this
yet. A neutron
port could be
associated
with a PCI
device or not,
which is a
common
feature, IMHO.
However, a ML2
driver may be
needed
specific to a
particular
SRIOV
technology. </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<blockquote style="border:none;border-left:solid #b5c4df 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US">During the Havana Release, we
introduced
Mellanox
Neutron plugin
that enables
networking via
SRIOV
pass-through
devices or
macvtap
interfaces.</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US">We want to integrate our solution
with PCI
pass-through
Nova support.
I will be
glad to share
more details
if you are
interested.</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Good to know that you
already have a
SRIOV
implementation.
I found out
some
information
online about
the mlnx
plugin, but
need more time
to get to know
it better. And
certainly I'm
interested in
knowing its
details.</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<blockquote style="border:none;border-left:solid #b5c4df 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US">The PCI pass-through networking
support is
planned to be
discussed
during the
summit:
</span><span style lang="EN-US"><a href="http://summit.openstack.org/cfp/details/129" target="_blank">http://summit.openstack.org/cfp/details/129</a>.
</span><span style="color:#1f497d" lang="EN-US">I
think it’s
worth to drill
down into more
detailed
proposal and
present it
during the
summit,
especially
since it
impacts both
nova and
neutron
projects.
</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">I agree. Maybe we can
steal some
time in that
discussion.</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<blockquote style="border:none;border-left:solid #b5c4df 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US">Would you be interested in
collaboration
on this
effort? Would
you be
interested to
exchange more
emails or set
an IRC/WebEx
meeting during
this week
before the
summit?</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Sure. If folks want to
discuss it
before the
summit, we can
schedule a
webex later
this week. Or
otherwise, we
can continue
the discussion
with email.</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<blockquote style="border:none;border-left:solid #b5c4df 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US">Regards,</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US">Irena</span><span style lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">From:</span></b><span style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">
Robert Li
(baoli) [<a href="mailto:baoli@cisco.com" target="_blank">mailto:baoli@cisco.com</a>]
<br>
<b>Sent:</b>
Friday,
October 25,
2013 11:16 PM<br>
<b>To:</b> <a href="mailto:prashant.upadhyaya@aricent.com" target="_blank">prashant.upadhyaya@aricent.com</a>;
Irena
Berezovsky;
<a href="mailto:yunhong.jiang@intel.com" target="_blank">yunhong.jiang@intel.com</a>; <a href="mailto:chris.friesen@windriver.com" target="_blank">
chris.friesen@windriver.com</a>; <a href="mailto:yongli.he@intel.com" target="_blank">yongli.he@intel.com</a><br>
<b>Cc:</b>
OpenStack
Development
Mailing List;
Brian Bowen
(brbowen);
Kyle Mestery
(kmestery);
Sandhya Dasu
(sadasu)<br>
<b>Subject:</b>
Re:
[openstack-dev]
[nova]
[neutron] PCI
pass-through
network
support</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
</div>
<p class="MsoNormal"><span style lang="EN-US"> <u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Hi Irena,</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">This is Robert Li from
Cisco Systems.
Recently, I
was tasked to
investigate
such support
for Cisco's
systems that
support
VM-FEX, which
is a SRIOV
technology
supporting
802-1Qbh. I
was able to
bring up nova
instances with
SRIOV
interfaces,
and establish
networking in
between the
instances that
employes the
SRIOV
interfaces.
Certainly,
this was
accomplished
with hacking
and some
manual
intervention.
Based on this
experience and
my study with
the two
existing nova
pci-passthrough
blueprints
that have been
implemented
and committed
into Havana (<a href="https://blueprints.launchpad.net/nova/+spec/pci-passthrough-base" target="_blank"><span style="color:rgb(0,51,170);font-size:9pt;text-decoration:none;font-family:Arial,sans-serif;background-repeat:initial initial">https://blueprints.launchpad.net/nova/+spec/pci-passthrough-base</span></a></span><span style="color:rgb(51,51,51);font-size:9pt;font-family:Arial,sans-serif;background-repeat:initial initial" lang="EN-US"> and<br>
</span><span style="color:rgb(0,51,170);font-size:9pt;font-family:Arial,sans-serif;background-repeat:initial initial" lang="EN-US"><a href="https://blueprints.launchpad.net/nova/+spec/pci-passthrough-libvirt" target="_blank"><span style="font-size:10.5pt">https://blueprints.launchpad.net/nova/+spec/pci-passthrough-libvirt</span></a>), </span><span style="font-size:10.5pt" lang="EN-US"> I registered a couple
of blueprints
(one on Nova
side, the
other on the
Neutron side):</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><a href="https://blueprints.launchpad.net/nova/+spec/pci-passthrough-sriov" target="_blank">https://blueprints.launchpad.net/nova/+spec/pci-passthrough-sriov</a></span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><a href="https://blueprints.launchpad.net/neutron/+spec/pci-passthrough-sriov" target="_blank">https://blueprints.launchpad.net/neutron/+spec/pci-passthrough-sriov</a></span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">in order to address
SRIOV support
in openstack. </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Please take a look at
them and see
if they make
sense, and let
me know any
comments and
questions. We
can also
discuss this
in the summit,
I suppose.</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">I noticed that there
is another
thread on this
topic, so copy
those folks
from that
thread as
well.</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">thanks,</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">Robert</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US">On 10/16/13 4:32 PM,
"Irena
Berezovsky"
<<a href="mailto:irenab@mellanox.com" target="_blank">irenab@mellanox.com</a>> wrote:</span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"> </span><span style lang="EN-US"><u></u><u></u></span></p>
</div>
<blockquote style="border:none;border-left:solid #b5c4df 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span style lang="EN-US">Hi,<u></u><u></u></span></p>
<p class="MsoNormal"><span style lang="EN-US">As one of the next steps for PCI
pass-through I
would like to
discuss is the
support for
PCI
pass-through
vNIC.<u></u><u></u></span></p>
<p class="MsoNormal"><span style lang="EN-US">While nova takes care of PCI
pass-through
device
resources
management
and VIF
settings,
neutron should
manage their
networking
configuration.<u></u><u></u></span></p>
<p class="MsoNormal"><span style lang="EN-US">I would like to register asummit
proposal to
discuss the
support for
PCI
pass-through
networking.<u></u><u></u></span></p>
<p class="MsoNormal"><span style lang="EN-US">I am not sure what would be the right
topic to
discuss the
PCI
pass-through
networking,
since it
involve both
nova and
neutron. <u></u><u></u></span></p>
<p class="MsoNormal"><span style lang="EN-US">There is already a session registered
by Yongli on
nova topic to
discuss the
PCI
pass-through
next steps.
<u></u><u></u></span></p>
<p class="MsoNormal"><span style lang="EN-US">I think PCI pass-through networking is
quite a big
topic and it
worth to have
a separate
discussion.<u></u><u></u></span></p>
<p class="MsoNormal"><span style lang="EN-US">Is there any other people who are
interested to
discuss it and
share their
thoughts and
experience?<u></u><u></u></span></p>
<p class="MsoNormal"><span style lang="EN-US"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style lang="EN-US">Regards,<u></u><u></u></span></p>
<p class="MsoNormal"><span style lang="EN-US">Irena<u></u><u></u></span></p>
<p class="MsoNormal"><span style lang="EN-US"> <u></u><u></u></span></p>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><span style="font-size:10.5pt" lang="EN-US"><u></u> <u></u></span></p>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</span>
</blockquote>
<br>
</div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>