<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">On 2013年12月17日 23:09, Ian Wells wrote:<br>
</div>
<blockquote
cite="mid:CAPoubz5XuA7WuXyRZEkOjyX36US5sQ3GfWtzY9jt0yiFq2tSww@mail.gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
<div dir="ltr">
<div>
<div>
<div>Reiterating from the IRC mneeting, largely, so
apologies.<br>
<br>
</div>
<div>Firstly, I disagree that <a moz-do-not-send="true"
href="https://wiki.openstack.org/wiki/PCI_passthrough_SRIOV_support">https://wiki.openstack.org/wiki/PCI_passthrough_SRIOV_support</a>
is an accurate reflection of the current state. It's a
very </div>
</div>
</div>
</div>
</blockquote>
do you really find anything is not what you want except the API
part? <br>
group device /fade out alias/ simplify the reguler expression for
address filter/ mark the device allocated the sriov? <br>
<br>
Yongli he<br>
<blockquote
cite="mid:CAPoubz5XuA7WuXyRZEkOjyX36US5sQ3GfWtzY9jt0yiFq2tSww@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>
<div>unilateral view, largely because the rest of us had
been focussing on the google document that we've been
using for weeks.<br>
</div>
<div><br>
Secondly, I totally disagree with this approach. This
assumes that description of the (cloud-internal, hardware)
details of each compute node is best done with data stored
centrally and driven by an API. I don't agree with either
of these points.<br>
<br>
</div>
Firstly, the best place to describe what's available on a
compute node is in the configuration on the compute node.
For instance, I describe which interfaces do what in Neutron
on the compute node. This is because when you're
provisioning nodes, that's the moment you know how you've
attached it to the network and what hardware you've put in
it and what you intend the hardware to be for - or
conversely your deployment puppet or chef or whatever knows
it, and Razor or MAAS has enumerated it, but the activities
are equivalent. Storing it centrally distances the compute
node from its descriptive information for no good purpose
that I can see and adds the complexity of having to go make
remote requests just to start up.<br>
<br>
</div>
Secondly, even if you did store this centrally, it's not clear
to me that an API is very useful. As far as I can see, the
need for an API is really the need to manage PCI device
flavors. If you want that to be API-managed, then the rest of
a (rather complex) API cascades from that one choice. Most of
the things that API lets you change (expressions describing
PCI devices) are the sort of thing that you set once and only
revisit when you start - for instance - deploying new hosts in
a different way.<br>
<br>
I at the parallel in Neutron provider networks. They're
config driven, largely on the compute hosts. Agents know what
ports on their machine (the hardware tie) are associated with
provider networks, by provider network name. The controller
takes 'neutron net-create ... --provider:network 'name'' and
uses that to tie a virtual network to the provider network
definition on each host. What we absolutely don't do is have
a complex admin API that lets us say 'in host aggregate 4,
provider network x (which I made earlier) is connected to
eth6'.<br>
<br>
-- <br>
</div>
Ian.<br>
<div>
<div><br>
</div>
</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On 17 December 2013 03:12, yongli he <span
dir="ltr"><<a moz-do-not-send="true"
href="mailto:yongli.he@intel.com" target="_blank">yongli.he@intel.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div>On 2013年12月16日 22:27, Robert Li (baoli) wrote:<br>
</div>
<blockquote type="cite">
<div>Hi Yongli,</div>
<div><br>
</div>
<div>The IRC meeting we have for PCI-Passthrough is the
forum for discussion on SR-IOV support in openstack. I
think the goal is to come up with a plan on both the
nova and neutron side in support of the SR-IOV, and
the current focus is on the nova side. Since you've
done a lot of work on it already, would you like to
lead tomorrow's discussion at UTC 1400? <br>
</div>
</blockquote>
<br>
Robert , you lead the meeting very well i enjoy you setup
every for us, keep going on it -:)<br>
<br>
I'd like to give you guy a summary of current state, let's
discuss it then. <br>
<a moz-do-not-send="true"
href="https://wiki.openstack.org/wiki/PCI_passthrough_SRIOV_support"
target="_blank">https://wiki.openstack.org/wiki/PCI_passthrough_SRIOV_support</a><br>
<br>
<br>
1) fade out alias ( i think this ok for all)<br>
2) white list became pic-flavor ( i think this ok for
all)<br>
3) address simply regular expression support: only * and
a number range is support [hex-hex]. ( i think this ok?)<br>
4) aggregate : now it's clear enough, and won't impact
SRIOV. ( i think this irrelevant to SRIOV now)<br>
<br>
<br>
5) SRIOV use case, if you suggest a use case, please
given a full example like this: [discuss: compare to other
solution]<br>
<br>
<ul
style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:start;font-style:normal;font-weight:normal;padding:0px;line-height:20px;color:rgb(51,51,51);text-transform:none;font-size:14px;white-space:normal;margin:0.3em
0px 0px 1.6em;font-family:'Arial Unicode
MS',Arial,sans-serif;word-spacing:0px">
<li style="line-height:20px">create a pci flavor for the
SRIOV</li>
</ul>
<pre style="padding:9.5px;font-family:Monaco,Menlo,Consolas,'Courier New',monospace;font-size:13px;color:rgb(51,51,51);border-top-left-radius:4px;border-top-right-radius:4px;border-bottom-right-radius:4px;border-bottom-left-radius:4px;display:block;margin:0px 0px 10px;line-height:20px;word-break:break-all;word-wrap:break-word;white-space:pre-wrap;background-color:rgb(245,245,245);border:1px solid rgba(0,0,0,0.14902);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px">
nova pci-flavor-create name 'vlan-SRIOV' description "xxxxx"
nova pci-flavor-update UUID set 'description'='xxxx' 'address'= '0000:01:*.7'
</pre>
<h5
style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:start;font-style:normal;font-weight:bold;line-height:20px;color:rgb(51,51,51);text-transform:none;font-size:14px;white-space:normal;margin:10px
0px;font-family:'Arial Unicode
MS',Arial,sans-serif;word-spacing:0px">
<br>
<span></span></h5>
<h5
style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:start;font-style:normal;font-weight:bold;line-height:20px;color:rgb(51,51,51);text-transform:none;font-size:14px;white-space:normal;margin:10px
0px;font-family:'Arial Unicode
MS',Arial,sans-serif;word-spacing:0px">
<span>Admin config SRIOV</span></h5>
<ul
style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:start;font-style:normal;font-weight:normal;padding:0px;line-height:20px;color:rgb(51,51,51);text-transform:none;font-size:14px;white-space:normal;margin:0.3em
0px 0px 1.6em;font-family:'Arial Unicode
MS',Arial,sans-serif;word-spacing:0px">
<li style="line-height:20px">create pci-flavor :</li>
</ul>
<pre style="padding:9.5px;font-family:Monaco,Menlo,Consolas,'Courier New',monospace;font-size:13px;color:rgb(51,51,51);border-top-left-radius:4px;border-top-right-radius:4px;border-bottom-right-radius:4px;border-bottom-left-radius:4px;display:block;margin:0px 0px 10px;line-height:20px;word-break:break-all;word-wrap:break-word;white-space:pre-wrap;background-color:rgb(245,245,245);border:1px solid rgba(0,0,0,0.14902);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px">
{"name": "privateNIC", "neutron-network-uuid": "uuid-1", ...}
{"name": "publicNIC", "neutron-network-uuid": "uuid-2", ...}
{"name": "smallGPU", "neutron-network-uuid": "", ...}
</pre>
<ul
style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:start;font-style:normal;font-weight:normal;padding:0px;line-height:20px;color:rgb(51,51,51);text-transform:none;font-size:14px;white-space:normal;margin:0.3em
0px 0px 1.6em;font-family:'Arial Unicode
MS',Arial,sans-serif;word-spacing:0px">
<li style="line-height:20px">set aggregate meta
according the flavors existed in the hosts</li>
</ul>
<p
style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:start;font-style:normal;font-weight:normal;line-height:20px;color:rgb(51,51,51);text-transform:none;font-size:14px;white-space:normal;margin:0px
0px 10px;font-family:'Arial Unicode
MS',Arial,sans-serif;word-spacing:0px">
flavor extra-specs, for a VM that gets two small GPUs
and VIFs attached from the above SRIOV NICs:</p>
<pre style="padding:9.5px;font-family:Monaco,Menlo,Consolas,'Courier New',monospace;font-size:13px;color:rgb(51,51,51);border-top-left-radius:4px;border-top-right-radius:4px;border-bottom-right-radius:4px;border-bottom-left-radius:4px;display:block;margin:0px 0px 10px;line-height:20px;word-break:break-all;word-wrap:break-word;white-space:pre-wrap;background-color:rgb(245,245,245);border:1px solid rgba(0,0,0,0.14902);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px">
nova aggregate-set-metadata pci-aware-group set 'pci-flavor'='smallGPU,oldGPU, privateNIC,privateNIC'
</pre>
<ul
style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:start;font-style:normal;font-weight:normal;padding:0px;line-height:20px;color:rgb(51,51,51);text-transform:none;font-size:14px;white-space:normal;margin:0.3em
0px 0px 1.6em;font-family:'Arial Unicode
MS',Arial,sans-serif;word-spacing:0px">
<li style="line-height:20px">create instance flavor for
sriov</li>
</ul>
<pre style="padding:9.5px;font-family:Monaco,Menlo,Consolas,'Courier New',monospace;font-size:13px;color:rgb(51,51,51);border-top-left-radius:4px;border-top-right-radius:4px;border-bottom-right-radius:4px;border-bottom-left-radius:4px;display:block;margin:0px 0px 10px;line-height:20px;word-break:break-all;word-wrap:break-word;white-space:pre-wrap;background-color:rgb(245,245,245);border:1px solid rgba(0,0,0,0.14902);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px">
nova flavor-key 100 set 'pci-flavor='1:privateNIC; 1: publicNIC; 2:smallGPU,oldGPU'
</pre>
<ul
style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:start;font-style:normal;font-weight:normal;padding:0px;line-height:20px;color:rgb(51,51,51);text-transform:none;font-size:14px;white-space:normal;margin:0.3em
0px 0px 1.6em;font-family:'Arial Unicode
MS',Arial,sans-serif;word-spacing:0px">
<li style="line-height:20px">User just specifies a
quantum port as normal:</li>
</ul>
<pre style="padding:9.5px;font-family:Monaco,Menlo,Consolas,'Courier New',monospace;font-size:13px;color:rgb(51,51,51);border-top-left-radius:4px;border-top-right-radius:4px;border-bottom-right-radius:4px;border-bottom-left-radius:4px;display:block;margin:0px 0px 10px;line-height:20px;word-break:break-all;word-wrap:break-word;white-space:pre-wrap;background-color:rgb(245,245,245);border:1px solid rgba(0,0,0,0.14902);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px">
nova boot --flavor "sriov-plus-two-gpu" --image img --nic net-id=uuid-2 --nic net-id=uuid-1 vm-name</pre>
<br>
<br>
Yongli <br>
<br>
<br>
<blockquote type="cite">
<div><br>
</div>
<div>Thanks,</div>
<div>Robert</div>
<div><br>
</div>
<span>
<div>
<div>On 12/11/13 8:09 PM, "He, Yongli" <<a
moz-do-not-send="true"
href="mailto:yongli.he@intel.com"
target="_blank">yongli.he@intel.com</a>>
wrote:</div>
</div>
<div><br>
</div>
<blockquote style="BORDER-LEFT:#b5c4df 5
solid;PADDING:0 0 0 5;MARGIN:0 0 0 5">
<div>
<div link="blue" vlink="purple" lang="ZH-CN">
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:#1f497d"></span><span
style="font-size:10.5pt;color:#1f497d"
lang="EN-US">Hi, all</span></p>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:#1f497d"
lang="EN-US">Please continue to foucs on
the blueprint, it change after reviewing.
And for this point:</span></p>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:#1f497d"
lang="EN-US"> </span></p>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"><br>
>5. flavor style for sriov: i just list
the flavor style in the design but for the
style <br>
> --nic <br>
> --pci-flavor
PowerfullNIC:1<br>
> still possible to work, so what's
the real impact to sriov from the flavor
design?</span></p>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span></p>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">>As
you can see from the log, Irena has some
strong opinions on this, and I tend to
agree with her. The problem we need to
solve is this: we need a means to
associate a nic (or port) with a PCI
device that is allocated out of a PCI
>flavor or a PCI group. We think that
we presented a complete solution in our
google doc.</span></p>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">It’s
not so clear, could you please list the
key point here. Btw, the blue print I sent
Monday had changed for this, please check.</span></p>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span></p>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span></p>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">Yongli
he</span></p>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:#1f497d"
lang="EN-US"> </span></p>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:#1f497d"
lang="EN-US"> </span></p>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:#1f497d"
lang="EN-US"> </span></p>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:#1f497d"
lang="EN-US"> </span></p>
<div>
<div style="border:none;border-top:solid
#b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span
style="font-size:10pt;font-family:Tahoma,sans-serif"
lang="EN-US">From:</span></b><span
style="font-size:10pt;font-family:Tahoma,sans-serif"
lang="EN-US"> Robert Li (baoli) [<a
moz-do-not-send="true"
href="mailto:baoli@cisco.com"
target="_blank">mailto:baoli@cisco.com</a>]
<br>
<b>Sent:</b> Wednesday, December 11,
2013 10:18 PM<br>
<b>To:</b> He, Yongli; Sandhya Dasu
(sadasu); OpenStack Development
Mailing List (not for usage
questions); Jiang, Yunhong; Irena
Berezovsky; <a moz-do-not-send="true"
href="mailto:prashant.upadhyaya@aricent.com" target="_blank">prashant.upadhyaya@aricent.com</a>;
<a moz-do-not-send="true"
href="mailto:chris.friesen@windriver.com"
target="_blank">chris.friesen@windriver.com</a>;
Itzik Brown; <a
moz-do-not-send="true"
href="mailto:john@johngarbutt.com"
target="_blank">john@johngarbutt.com</a><br>
<b>Subject:</b> Re: [openstack-dev]
[nova] [neutron] PCI pass-through
network support</span></p>
</div>
</div>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">Hi
Yongli,</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">Thank
you very much for sharing the Wiki with
us on Monday so that we have a better
understanding on your ideas and
thoughts. Please see embedded comments.</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">--Robert</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">On
12/10/13 8:35 PM, "yongli he" <<a
moz-do-not-send="true"
href="mailto:yongli.he@intel.com"
target="_blank">yongli.he@intel.com</a>>
wrote:</span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span></p>
</div>
<blockquote
style="border:none;border-left:solid #b5c4df
4.5pt;padding:0cm 0cm 0cm
4.0pt;margin-left:3.75pt;margin-right:0cm">
<div>
<div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US">On 2013</span><span
style="font-size:10.5pt;font-family:PMingLiU,serif"
lang="ZH-TW">年</span><span
style="font-size:10.5pt"
lang="EN-US">12</span><span
style="font-size:10.5pt;font-family:PMingLiU,serif"
lang="ZH-TW">月</span><span
style="font-size:10.5pt"
lang="EN-US">10</span><span
style="font-size:10.5pt;font-family:PMingLiU,serif"
lang="ZH-TW">日</span><span
style="font-size:10.5pt"
lang="EN-US"> 22:41, Sandhya Dasu
(sadasu) wrote:</span></p>
</div>
<blockquote
style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US">Hi,</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"> I am trying to
resurrect this email thread
since discussions have split
between several threads and is
becoming hard to keep track.</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US">An update:</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US">New PCI Passthrough
meeting time: Tuesdays UTC 1400.</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US">New PCI flavor
proposal from Nova:</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"><a
moz-do-not-send="true"
href="https://wiki.openstack.org/wiki/PCI_configration_Database_and_API#Take_advantage_of_host_aggregate_.28T.B.D.29"
target="_blank">https://wiki.openstack.org/wiki/PCI_configration_Database_and_API#Take_advantage_of_host_aggregate_.28T.B.D.29</a></span></p>
</div>
</blockquote>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US">Hi, all<br>
sorry for miss the meeting, i was
seeking John at that time. from the
log i saw some concern about new
design, i list them there and try
to clarify it per my opinion:<br>
<br>
1. configuration going to
deprecated: this might impact
SRIOV. if possible, please list
what kind of impact make to you. </span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">Regarding
the nova API pci-flavor-update, we had a
face-to-face discussion over use of a
nova API to provision/define/configure
PCI passthrough list during the
ice-house summit. I kind of like the
idea initially. As you can see from the
meeting log, however, I later thought
that in a distributed system, using a
centralized API to define resources per
compute node, which could come and go
any time, doesn't seem to provide any
significant benefit. This is the reason
that I didn't mention it in our google
doc <a moz-do-not-send="true"
href="https://docs.google.com/document/d/1EMwDg9J8zOxzvTnQJ9HwZdiotaVstFWKIuKrPse6JOs/edit#"
target="_blank">https://docs.google.com/document/d/1EMwDg9J8zOxzvTnQJ9HwZdiotaVstFWKIuKrPse6JOs/edit#</a></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">If
you agree that pci-flavor and pci-group
is kind of the same thing, then we agree
with you that the pci-flavor-create API
is needed. Since pci-flavor or pci-group
is global, then such an API can be used
for resource registration/validation on
nova server. In addition, it can be used
to facilitate the display of PCI devices
per node, per group, or in the entire
cloud, etc.</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span></p>
</div>
<blockquote
style="border:none;border-left:solid #b5c4df
4.5pt;padding:0cm 0cm 0cm
4.0pt;margin-left:3.75pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"><br>
<br>
2. <baoli>So the API seems to
be combining the whitelist +
pci-group<br>
yeah, it's actually almost same
thing, 'flavor' 'pci-group' or
'group'. the real different is this
flavor going to deprecated the
alias, and combine tight to
aggregate or flavor. </span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">Well,
with pci-group, we recommended to
deprecate the PCI alias because we think
it is redundant. </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">We
think that specification of PCI
requirement in the flavor's extra spec
is still needed as it's a generic means
to allocate PCI devices. In addition, it
can be used as properties in the host
aggregate as well. </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span></p>
</div>
<blockquote
style="border:none;border-left:solid #b5c4df
4.5pt;padding:0cm 0cm 0cm
4.0pt;margin-left:3.75pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"><br>
<br>
3. feature: <br>
this design is not to say the
feature is not work, but changed.
if auto discovery feature is
possible, we got 'feature' form the
device, then use the feature to
define the pci-flavor. it's also
possible create default pci-flavor
for this. so the feature concept
will be impact, my feeling, we
should given a separated bp for
feature, and not in this round
change, so here we only thing is
keep the feature is possible. </span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">I
think that it's ok to have separate BPs.
But we think that auto discovery is an
essential part of the design, and
therefore it should be implemented with
more helping hands. </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span></p>
</div>
<blockquote
style="border:none;border-left:solid #b5c4df
4.5pt;padding:0cm 0cm 0cm
4.0pt;margin-left:3.75pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"><br>
<br>
4. address regular expression: i'm
fine with the wild-match style.</span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">Sounds
good. One side node is that I noticed
that the driver for intel 82576 cards
has a strange slot assignment scheme. So
the final definition of it may need to
accommodate that as well.</span></p>
</div>
<blockquote
style="border:none;border-left:solid #b5c4df
4.5pt;padding:0cm 0cm 0cm
4.0pt;margin-left:3.75pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"><br>
<br>
5. flavor style for sriov: i just
list the flavor style in the design
but for the style <br>
--nic <br>
--pci-flavor
PowerfullNIC:1<br>
still possible to work, so what's
the real impact to sriov from the
flavor design?</span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">As
you can see from the log, Irena has some
strong opinions on this, and I tend to
agree with her. The problem we need to
solve is this: we need a means to
associate a nic (or port) with a PCI
device that is allocated out of a PCI
flavor or a PCI group. We think that we
presented a complete solution in our
google doc.</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">At
this point, I really believe that we
should combine our efforts and ideas. As
far as how many BPs are needed, it
should be a trivial matter after we have
agreed on a complete solution.</span></p>
</div>
<blockquote
style="border:none;border-left:solid #b5c4df
4.5pt;padding:0cm 0cm 0cm
4.0pt;margin-left:3.75pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"><br>
<br>
Yongli He<br>
<br>
<br>
<br>
</span></p>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US">Thanks,</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US">Sandhya</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"> </span></p>
</div>
<div style="border:none;border-top:solid
#b5c4df 1.0pt;padding:3.0pt 0cm 0cm
0cm">
<p class="MsoNormal"><b><span style=""
lang="EN-US">From: </span></b><span
style="" lang="EN-US">Sandhya Dasu
<<a moz-do-not-send="true"
href="mailto:sadasu@cisco.com"
target="_blank">sadasu@cisco.com</a>><br>
<b>Reply-To: </b>"OpenStack
Development Mailing List (not for
usage questions)" <<a
moz-do-not-send="true"
href="mailto:openstack-dev@lists.openstack.org"
target="_blank">openstack-dev@lists.openstack.org</a>><br>
<b>Date: </b>Thursday, November
7, 2013 9:44 PM<br>
<b>To: </b>"OpenStack Development
Mailing List (not for usage
questions)" <<a
moz-do-not-send="true"
href="mailto:openstack-dev@lists.openstack.org"
target="_blank">openstack-dev@lists.openstack.org</a>>,
"Jiang, Yunhong" <<a
moz-do-not-send="true"
href="mailto:yunhong.jiang@intel.com"
target="_blank">yunhong.jiang@intel.com</a>>,
"Robert Li (baoli)" <<a
moz-do-not-send="true"
href="mailto:baoli@cisco.com"
target="_blank">baoli@cisco.com</a>>,
Irena Berezovsky <<a
moz-do-not-send="true"
href="mailto:irenab@mellanox.com"
target="_blank">irenab@mellanox.com</a>>,
"<a moz-do-not-send="true"
href="mailto:prashant.upadhyaya@aricent.com"
target="_blank">prashant.upadhyaya@aricent.com</a>"
<<a moz-do-not-send="true"
href="mailto:prashant.upadhyaya@aricent.com"
target="_blank">prashant.upadhyaya@aricent.com</a>>,
"<a moz-do-not-send="true"
href="mailto:chris.friesen@windriver.com"
target="_blank">chris.friesen@windriver.com</a>"
<<a moz-do-not-send="true"
href="mailto:chris.friesen@windriver.com"
target="_blank">chris.friesen@windriver.com</a>>,
"He, Yongli" <<a
moz-do-not-send="true"
href="mailto:yongli.he@intel.com"
target="_blank">yongli.he@intel.com</a>>,
Itzik Brown <<a
moz-do-not-send="true"
href="mailto:ItzikB@mellanox.com"
target="_blank">ItzikB@mellanox.com</a>><br>
<b>Subject: </b>Re:
[openstack-dev] [nova] [neutron]
PCI pass-through network support</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"> </span></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US">Hi,</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"> The
discussions during the summit
were very productive. Now, we
are ready to setup our IRC
meeting.</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US">Here are some
slots that look like they
might work for us.</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US">1. Wed 2 – 3 pm
UTC.</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US">2. Thursday 12 –
1 pm UTC.</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US">3. Thursday 7 –
8pm UTC.</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US">Please vote.</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US">Thanks,</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US">Sandhya</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"> </span></p>
</div>
<div
style="border:none;border-top:solid
#b5c4df 1.0pt;padding:3.0pt 0cm
0cm 0cm">
<p class="MsoNormal"><b><span
style="" lang="EN-US">From:
</span></b><span style=""
lang="EN-US">Sandhya Dasu <<a
moz-do-not-send="true"
href="mailto:sadasu@cisco.com"
target="_blank">sadasu@cisco.com</a>><br>
<b>Reply-To: </b>"OpenStack
Development Mailing List (not
for usage questions)" <<a
moz-do-not-send="true"
href="mailto:openstack-dev@lists.openstack.org"
target="_blank">openstack-dev@lists.openstack.org</a>><br>
<b>Date: </b>Tuesday,
November 5, 2013 12:03 PM<br>
<b>To: </b>"OpenStack
Development Mailing List (not
for usage questions)" <<a
moz-do-not-send="true"
href="mailto:openstack-dev@lists.openstack.org"
target="_blank">openstack-dev@lists.openstack.org</a>>,
"Jiang, Yunhong" <<a
moz-do-not-send="true"
href="mailto:yunhong.jiang@intel.com"
target="_blank">yunhong.jiang@intel.com</a>>,
"Robert Li (baoli)" <<a
moz-do-not-send="true"
href="mailto:baoli@cisco.com"
target="_blank">baoli@cisco.com</a>>,
Irena Berezovsky <<a
moz-do-not-send="true"
href="mailto:irenab@mellanox.com"
target="_blank">irenab@mellanox.com</a>>,
"<a moz-do-not-send="true"
href="mailto:prashant.upadhyaya@aricent.com"
target="_blank">prashant.upadhyaya@aricent.com</a>"
<<a moz-do-not-send="true"
href="mailto:prashant.upadhyaya@aricent.com" target="_blank">prashant.upadhyaya@aricent.com</a>>,
"<a moz-do-not-send="true"
href="mailto:chris.friesen@windriver.com"
target="_blank">chris.friesen@windriver.com</a>"
<<a moz-do-not-send="true"
href="mailto:chris.friesen@windriver.com" target="_blank">chris.friesen@windriver.com</a>>,
"He, Yongli" <<a
moz-do-not-send="true"
href="mailto:yongli.he@intel.com"
target="_blank">yongli.he@intel.com</a>>,
Itzik Brown <<a
moz-do-not-send="true"
href="mailto:ItzikB@mellanox.com"
target="_blank">ItzikB@mellanox.com</a>><br>
<b>Subject: </b>Re:
[openstack-dev] [nova]
[neutron] PCI pass-through
network support</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"> </span></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US">Just to
clarify, the discussion is
planned for 10 AM
Wednesday morning at the
developer's lounge.</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US">Thanks,</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US">Sandhya</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"> </span></p>
</div>
<div
style="border:none;border-top:solid
#b5c4df 1.0pt;padding:3.0pt
0cm 0cm 0cm">
<p class="MsoNormal"><b><span
style="" lang="EN-US">From:
</span></b><span style=""
lang="EN-US">Sandhya Dasu
<<a
moz-do-not-send="true"
href="mailto:sadasu@cisco.com"
target="_blank">sadasu@cisco.com</a>><br>
<b>Reply-To: </b>"OpenStack
Development Mailing List
(not for usage questions)"
<<a
moz-do-not-send="true"
href="mailto:openstack-dev@lists.openstack.org"
target="_blank">openstack-dev@lists.openstack.org</a>><br>
<b>Date: </b>Tuesday,
November 5, 2013 11:38 AM<br>
<b>To: </b>"OpenStack
Development Mailing List
(not for usage questions)"
<<a
moz-do-not-send="true"
href="mailto:openstack-dev@lists.openstack.org"
target="_blank">openstack-dev@lists.openstack.org</a>>,
"Jiang, Yunhong" <<a
moz-do-not-send="true"
href="mailto:yunhong.jiang@intel.com"
target="_blank">yunhong.jiang@intel.com</a>>,
"Robert Li (baoli)" <<a
moz-do-not-send="true"
href="mailto:baoli@cisco.com"
target="_blank">baoli@cisco.com</a>>,
Irena Berezovsky <<a
moz-do-not-send="true"
href="mailto:irenab@mellanox.com"
target="_blank">irenab@mellanox.com</a>>,
"<a moz-do-not-send="true"
href="mailto:prashant.upadhyaya@aricent.com" target="_blank">prashant.upadhyaya@aricent.com</a>"
<<a
moz-do-not-send="true"
href="mailto:prashant.upadhyaya@aricent.com"
target="_blank">prashant.upadhyaya@aricent.com</a>>,
"<a moz-do-not-send="true"
href="mailto:chris.friesen@windriver.com" target="_blank">chris.friesen@windriver.com</a>"
<<a
moz-do-not-send="true"
href="mailto:chris.friesen@windriver.com"
target="_blank">chris.friesen@windriver.com</a>>,
"He, Yongli" <<a
moz-do-not-send="true"
href="mailto:yongli.he@intel.com"
target="_blank">yongli.he@intel.com</a>>,
Itzik Brown <<a
moz-do-not-send="true"
href="mailto:ItzikB@mellanox.com"
target="_blank">ItzikB@mellanox.com</a>><br>
<b>Subject: </b>Re:
[openstack-dev] [nova]
[neutron] PCI pass-through
network support</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"> </span></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><b><span
style=""
lang="EN-US">Hi,</span></b><span
style="font-size:10.5pt" lang="EN-US"></span></p>
</div>
<div>
<p class="MsoNormal"><b><span
style=""
lang="EN-US"> We
are planning to have
a discussion at the
developer's lounge
tomorrow morning at
10:00 am. Please
feel free to drop by
if you are
interested.</span></b><span
style="font-size:10.5pt" lang="EN-US"></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><b><span
style=""
lang="EN-US">Thanks,</span></b><span
style="font-size:10.5pt" lang="EN-US"></span></p>
</div>
<div>
<p class="MsoNormal"><b><span
style=""
lang="EN-US">Sandhya</span></b><span
style="font-size:10.5pt" lang="EN-US"></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><b><span
style=""
lang="EN-US">From: </span></b><span
style="" lang="EN-US"><Jiang>,
Yunhong <</span><span
style="font-size:10.5pt" lang="EN-US"><a moz-do-not-send="true"
href="mailto:yunhong.jiang@intel.com"
target="_blank"><span
style="font-size:11.0pt">yunhong.jiang@intel.com</span></a></span><span
style="" lang="EN-US">></span><span
style="font-size:10.5pt" lang="EN-US"></span></p>
</div>
<div>
<div>
<div
style="border:none;border-top:solid
#b5c4df
1.0pt;padding:3.0pt
0cm 0cm 0cm">
<p class="MsoNormal"><b><span
style=""
lang="EN-US">Date:
</span></b><span
style=""
lang="EN-US">Thursday,
October 31, 2013
6:21 PM<br>
<b>To: </b>"Robert
Li (baoli)" <<a
moz-do-not-send="true" href="mailto:baoli@cisco.com" target="_blank">baoli@cisco.com</a>>,
Irena Berezovsky
<<a
moz-do-not-send="true"
href="mailto:irenab@mellanox.com" target="_blank">irenab@mellanox.com</a>>,
"<a
moz-do-not-send="true"
href="mailto:prashant.upadhyaya@aricent.com" target="_blank">prashant.upadhyaya@aricent.com</a>"
<<a
moz-do-not-send="true"
href="mailto:prashant.upadhyaya@aricent.com" target="_blank">prashant.upadhyaya@aricent.com</a>>,
"<a
moz-do-not-send="true"
href="mailto:chris.friesen@windriver.com" target="_blank">chris.friesen@windriver.com</a>"
<<a
moz-do-not-send="true"
href="mailto:chris.friesen@windriver.com" target="_blank">chris.friesen@windriver.com</a>>,
"He, Yongli" <<a
moz-do-not-send="true" href="mailto:yongli.he@intel.com" target="_blank">yongli.he@intel.com</a>>,
Itzik Brown <<a
moz-do-not-send="true" href="mailto:ItzikB@mellanox.com" target="_blank">ItzikB@mellanox.com</a>><br>
<b>Cc: </b>OpenStack
Development
Mailing List <<a
moz-do-not-send="true" href="mailto:openstack-dev@lists.openstack.org"
target="_blank">openstack-dev@lists.openstack.org</a>>,
"Brian Bowen
(brbowen)" <<a
moz-do-not-send="true" href="mailto:brbowen@cisco.com" target="_blank">brbowen@cisco.com</a>>,
"Kyle Mestery
(kmestery)" <<a
moz-do-not-send="true" href="mailto:kmestery@cisco.com" target="_blank">kmestery@cisco.com</a>>,
Sandhya Dasu <<a
moz-do-not-send="true" href="mailto:sadasu@cisco.com" target="_blank">sadasu@cisco.com</a>><br>
<b>Subject: </b>RE:
[openstack-dev]
[nova] [neutron]
PCI pass-through
network support</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:#1f497d" lang="EN-US">Robert, I think your
change request
for pci alias
should be
covered by the
extra infor
enhancement. </span><span
style=""
lang="EN-US"><a
moz-do-not-send="true"
href="https://blueprints.launchpad.net/nova/+spec/pci-extra-info"
target="_blank">https://blueprints.launchpad.net/nova/+spec/pci-extra-info</a>
and Yongli is
working on it.</span></p>
<p class="MsoNormal"><span
style=""
lang="EN-US"> </span></p>
<p class="MsoNormal"><span
style=""
lang="EN-US">I’m
not sure how the
port profile is
passed to the
connected
switch, is it a
Cisco VMEFX
specific method
or libvirt
method? Sorry
I’m not well on
network side.</span></p>
<p class="MsoNormal"><span
style=""
lang="EN-US"> </span></p>
<p class="MsoNormal"><span
style=""
lang="EN-US">--jyh</span></p>
<p class="MsoNormal"><span
style="font-size:10.5pt;color:#1f497d" lang="EN-US"> </span><span
style=""
lang="EN-US"></span></p>
<div
style="border:none;border-left:solid
blue
1.5pt;padding:0cm
0cm 0cm 4.0pt">
<div>
<div
style="border:none;border-top:solid
#b5c4df
1.0pt;padding:3.0pt
0cm 0cm 0cm">
<p
class="MsoNormal"><b><span
style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">From:</span></b><span
style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">
Robert Li
(baoli) [<a
moz-do-not-send="true"
href="mailto:baoli@cisco.com" target="_blank">mailto:baoli@cisco.com</a>]
<br>
<b>Sent:</b>
Wednesday,
October 30,
2013 10:13 AM<br>
<b>To:</b>
Irena
Berezovsky;
Jiang,
Yunhong; <a
moz-do-not-send="true"
href="mailto:prashant.upadhyaya@aricent.com" target="_blank">
prashant.upadhyaya@aricent.com</a>; <a moz-do-not-send="true"
href="mailto:chris.friesen@windriver.com"
target="_blank">
chris.friesen@windriver.com</a>; He, Yongli; Itzik Brown<br>
<b>Cc:</b>
OpenStack
Development
Mailing List;
Brian Bowen
(brbowen);
Kyle Mestery
(kmestery);
Sandhya Dasu
(sadasu)<br>
<b>Subject:</b>
Re:
[openstack-dev]
[nova]
[neutron] PCI
pass-through
network
support</span><span
style=""
lang="EN-US"></span></p>
</div>
</div>
<p
class="MsoNormal"><span
style=""
lang="EN-US"> </span></p>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">Hi,</span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">Regarding physical network
mapping, This
is what I
thought. </span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">consider the following scenarios:</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> 1. a compute node with SRIOV
only
interfaces
attached to a
physical
network. the
node is
connected to
one upstream
switch </span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> 2. a compute node with both
SRIOV
interfaces and
non-SRIOV
interfaces
attached to a
physical
network. the
node is
connected to
one upstream
switch</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> 3. in addition to case 1
&2, a
compute node
may have
multiple vNICs
that are
connected to
different
upstream
switches. </span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">CASE 1:</span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> -- the mapping from a virtual
network (in
terms of
neutron) to a
physical
network is
actually done
by binding a
port profile
to a neutron
port. With
cisco's
VM-FEX, a port
profile is
associated
with one or
multiple
vlans. Once
the neutron
port is bound
with this
port-profile
in the
upstream
switch, it's
effectively
plugged into
the physical
network. </span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> -- since the compute node is
connected to
one upstream
switch, the
existing nova
PCI alias will
be sufficient.
For example,
one can boot a
Nova instance
that is
attached to a
SRIOV port
with the
following
command:</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> nova boot —flavor
m1.large
—image
<image-id>
--nic
net-id=<net>,pci-alias=<alias>,sriov=<direct|macvtap>,port-profile=<profile></span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> the net-id will be useful in
terms of
allocating IP
address,
enable dhcp,
etc that is
associated
with the
network.</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">-- the pci-alias specified in the
nova boot
command is
used to create
a PCI request
for scheduling
purpose. a PCI
device is
bound to a
neutron port
during the
instance build
time in the
case of nova
boot. Before
invoking the
neutron API to
create a port,
an allocated
PCI device out
of a PCI alias
will be
located from
the PCI device
list object.
This device
info among
other
information
will be sent
to neutron to
create the
port. </span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">CASE 2:</span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">-- Assume that OVS is used for the
non-SRIOV
interfaces. An
example of
configuration
with ovs
plugin would
look like:</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> bridge_mappings =
physnet1:br-vmfex</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> network_vlan_ranges =
physnet1:15:17</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> tenant_network_type =
vlan</span><span
style=""
lang="EN-US"></span></p>
</div>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> When a neutron network is
created, a
vlan is either
allocated or
specified in
the neutron
net-create
command.
Attaching a
physical
interface to
the bridge (in
the above
example
br-vmfex) is
an
administrative
task. </span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">-- to create a Nova instance with
non-SRIOV
port:</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> nova boot —flavor
m1.large
—image
<image-id>
--nic
net-id=<net></span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">-- to create a Nova instance with
SRIOV port:</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> nova boot —flavor
m1.large
—image
<image-id>
--nic
net-id=<net>,pci-alias=<alias>,sriov=<direct|macvtap>,port-profile=<profile></span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> it's essentially the same as
in the first
case. But
since the
net-id is
already
associated
with a vlan,
the vlan
associated
with the
port-profile
must be
identical to
that vlan.
This has to be
enforced by
neutron.</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> again, since the node is
connected to
one upstream
switch, the
existing nova
PCI alias
should be
sufficient. </span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">CASE 3:</span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">-- A compute node might be
connected to
multiple
upstream
switches, with
each being a
separate
network. This
means SRIOV
PFs/VFs are
already
implicitly
associated
with physical
networks. In
the none-SRIOV
case, a
physical
interface is
associated
with a
physical
network by
plugging it
into that
network, and
attaching this
interface to
the ovs bridge
that
represents
this physical
network on the
compute node.
In the SRIOV
case, we need
a way to group
the SRIOV VFs
that belong to
the same
physical
networks. The
existing nova
PCI alias is
to facilitate
PCI device
allocation by
associating
<product_id,
vendor_id>
with an alias
name. This
will no longer
be sufficient.
But it can be
enhanced to
achieve our
goal. For
example, the
PCI device
domain, bus
(if their
mapping to
vNIC is fixed
across boot)
may be added
into the
alias, and the
alias name
should be
corresponding
to a list of
tuples. </span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">Another consideration is that a VF
or PF might be
used on the
host for other
purposes. For
example, it's
possible for a
neutron DHCP
server to be
bound with a
VF. Therefore,
there needs a
method to
exclude some
VFs from a
group. One
way is to
associate an
exclude list
with an alias.</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">The enhanced PCI alias can be used
to support
features other
than neutron
as well.
Essentially, a
PCI alias can
be defined as
a group of PCI
devices
associated
with a
feature. I'd
think that
this should be
addressed with
a separate
blueprint.</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">Thanks,</span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">Robert</span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">On 10/30/13 12:59 AM, "Irena
Berezovsky"
<<a
moz-do-not-send="true"
href="mailto:irenab@mellanox.com" target="_blank">irenab@mellanox.com</a>>
wrote:</span><span
style=""
lang="EN-US"></span></p>
</div>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<blockquote
style="border:none;border-left:solid
#b5c4df
4.5pt;padding:0cm
0cm 0cm
4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">Hi,</span><span style="" lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">Please see my answers inline</span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US"> </span><span style="" lang="EN-US"></span></p>
<div>
<div
style="border:none;border-top:solid
#b5c4df
1.0pt;padding:3.0pt
0cm 0cm 0cm">
<p
class="MsoNormal"><b><span
style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">From:</span></b><span
style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">
Jiang, Yunhong
[<a
moz-do-not-send="true"
href="mailto:yunhong.jiang@intel.com" target="_blank">mailto:yunhong.jiang@intel.com</a>]
<br>
<b>Sent:</b>
Tuesday,
October 29,
2013 10:17 PM<br>
<b>To:</b>
Irena
Berezovsky;
Robert Li
(baoli); <a
moz-do-not-send="true"
href="mailto:prashant.upadhyaya@aricent.com" target="_blank">
prashant.upadhyaya@aricent.com</a>; <a moz-do-not-send="true"
href="mailto:chris.friesen@windriver.com"
target="_blank">
chris.friesen@windriver.com</a>; He, Yongli; Itzik Brown<br>
<b>Cc:</b>
OpenStack
Development
Mailing List;
Brian Bowen
(brbowen);
Kyle Mestery
(kmestery);
Sandhya Dasu
(sadasu)<br>
<b>Subject:</b>
RE:
[openstack-dev]
[nova]
[neutron] PCI
pass-through
network
support</span><span
style=""
lang="EN-US"></span></p>
</div>
</div>
<p
class="MsoNormal"><span
style=""
lang="EN-US"> </span></p>
<p
class="MsoNormal"><span
style="font-size:10.5pt;color:#1f497d" lang="EN-US">Your explanation of
the virtual
network and
physical
network is
quite clear
and should
work well. We
need change
nova code to
achieve it,
including get
the physical
network for
the virtual
network,
passing the
physical
network
requirement to
the filter
properties
etc. </span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><b><i><span
style="font-size:10.5pt;color:#1f497d" lang="EN-US">[IrenaB] </span></i></b><span
style="font-size:10.5pt;color:#1f497d" lang="EN-US"> The physical
network is
already
available to
nova at
networking/nova/api
at as virtual
network
attribute, it
then passed to
the VIF
driver. We
will push soon
the fix to:</span><span
style=""
lang="EN-US"><a
moz-do-not-send="true"
href="https://bugs.launchpad.net/nova/+bug/1239606"
target="_blank">https://bugs.launchpad.net/nova/+bug/1239606</a></span><span
style="font-size:10.5pt;color:#1f497d" lang="EN-US"> ; which will
provide
general
support for
getting this
information.</span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="font-size:10.5pt;color:#1f497d" lang="EN-US"> </span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="font-size:10.5pt;color:#1f497d" lang="EN-US">For your port
method, so you
mean we are
sure to
passing
network id to
‘nova boot’
and nova will
create the
port during VM
boot, am I
right? Also,
how can nova
knows that it
need allocate
the PCI device
for the port?
I’d suppose
that in SR-IOV
NIC
environment,
user don’t
need specify
the PCI
requirement.
Instead, the
PCI
requirement
should come
from the
network
configuration
and image
property. Or
you think user
still need
passing flavor
with pci
request?</span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><b><i><span
style="color:#1f497d" lang="EN-US">[IrenaB] There are two way to apply
port method.
One is to pass
network id on
nova boot and
use default
type as chosen
in the neutron
config file
for vnic type.
Other way is
to define port
with required
vnic type and
other
properties if
applicable,
and run ‘nova
boot’ with
port id
argument.
Going forward
with nova
support for
PCI devices
awareness, we
do need a way
impact
scheduler
choice to land
VM on suitable
Host with
available PC
device that
has the
required
connectivity.</span></i></b><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="font-size:10.5pt;color:#1f497d" lang="EN-US"> </span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="font-size:10.5pt;color:#1f497d" lang="EN-US">--jyh</span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="font-size:10.5pt;color:#1f497d" lang="EN-US"> </span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="font-size:10.5pt;color:#1f497d" lang="EN-US"> </span><span
style=""
lang="EN-US"></span></p>
<div
style="border:none;border-left:solid
blue
1.5pt;padding:0cm
0cm 0cm 4.0pt">
<div>
<div
style="border:none;border-top:solid
#b5c4df
1.0pt;padding:3.0pt
0cm 0cm 0cm">
<p
class="MsoNormal"><b><span
style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">From:</span></b><span
style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US"> Irena
Berezovsky [<a
moz-do-not-send="true" href="mailto:irenab@mellanox.com" target="_blank">mailto:irenab@mellanox.com</a>]
<br>
<b>Sent:</b>
Tuesday,
October 29,
2013 3:17 AM<br>
<b>To:</b>
Jiang,
Yunhong;
Robert Li
(baoli); <a
moz-do-not-send="true"
href="mailto:prashant.upadhyaya@aricent.com" target="_blank">
prashant.upadhyaya@aricent.com</a>; <a moz-do-not-send="true"
href="mailto:chris.friesen@windriver.com"
target="_blank">
chris.friesen@windriver.com</a>; He, Yongli; Itzik Brown<br>
<b>Cc:</b>
OpenStack
Development
Mailing List;
Brian Bowen
(brbowen);
Kyle Mestery
(kmestery);
Sandhya Dasu
(sadasu)<br>
<b>Subject:</b>
RE:
[openstack-dev]
[nova]
[neutron] PCI
pass-through
network
support</span><span
style=""
lang="EN-US"></span></p>
</div>
</div>
<p
class="MsoNormal"><span
style=""
lang="EN-US"> </span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">Hi Jiang, Robert,</span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">IRC meeting option works for me. </span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">If I understand your question below,
you are
looking for a
way to tie up
between
requested
virtual
network(s) and
requested PCI
device(s). The
way we did it
in our
solution is
to map a
provider:physical_network
to an
interface that
represents the
Physical
Function.
Every virtual
network is
bound to the
provider:physical_network,
so the PCI
device should
be allocated
based on this
mapping. We
can map a PCI
alias to the
provider:physical_network.</span><span style="" lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US"> </span><span style="" lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">Another topic to discuss is where the
mapping
between
neutron port
and PCI device
should be
managed. One
way to solve
it, is to
propagate the
allocated PCI
device details
to neutron on
port creation.
</span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">In case there is no qbg/qbh support,
VF networking
configuration
should be
applied
locally on the
Host.</span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">The question is when and how to apply
networking
configuration
on the PCI
device? </span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">We see the following options:</span><span
style=""
lang="EN-US"></span></p>
<p
style="margin-left:38.7pt">
<span style=""
lang="EN-US"><span>‧<span>
</span></span></span><span
style="color:#1f497d" lang="EN-US">it can be done on port creation. </span><span
style=""
lang="EN-US"></span></p>
<p
style="margin-left:38.7pt">
<span style=""
lang="EN-US"><span>‧<span>
</span></span></span><span
style="color:#1f497d" lang="EN-US">It can be done when nova VIF driver
is called for
vNIC plugging.
This will
require to
have all
networking
configuration
available to
the VIF driver
or send
request to the
neutron server
to obtain it.
</span><span
style=""
lang="EN-US"></span></p>
<p
style="margin-left:38.7pt">
<span style=""
lang="EN-US"><span>‧<span>
</span></span></span><span
style="color:#1f497d" lang="EN-US">It can be done by having a dedicated
L2 neutron
agent on each
Host that
scans for
allocated PCI
devices and
then retrieves
networking
configuration
from the
server and
configures the
device. The
agent will be
also
responsible
for managing
update
requests
coming from
the neutron
server. </span><span
style=""
lang="EN-US"></span></p>
<p
style="margin-left:38.7pt"><span
style="color:#1f497d" lang="EN-US"> </span><span style="" lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">For macvtap vNIC type assignment, the
networking
configuration
can be applied
by a dedicated
L2 neutron
agent. </span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US"> </span><span style="" lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">BR,</span><span style="" lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">Irena</span><span style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US"> </span><span style="" lang="EN-US"></span></p>
<div>
<div
style="border:none;border-top:solid
#b5c4df
1.0pt;padding:3.0pt
0cm 0cm 0cm">
<p
class="MsoNormal"><b><span
style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">From:</span></b><span
style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">
Jiang, Yunhong
[<a
moz-do-not-send="true"
href="mailto:yunhong.jiang@intel.com" target="_blank">mailto:yunhong.jiang@intel.com</a>]
<br>
<b>Sent:</b>
Tuesday,
October 29,
2013 9:04 AM</span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US"><br>
<b>To:</b>
Robert Li
(baoli); Irena
Berezovsky; <a
moz-do-not-send="true" href="mailto:prashant.upadhyaya@aricent.com"
target="_blank">
prashant.upadhyaya@aricent.com</a>; <a moz-do-not-send="true"
href="mailto:chris.friesen@windriver.com"
target="_blank">
chris.friesen@windriver.com</a>; He, Yongli; Itzik Brown<br>
<b>Cc:</b>
OpenStack
Development
Mailing List;
Brian Bowen
(brbowen);
Kyle Mestery
(kmestery);
Sandhya Dasu
(sadasu)<br>
<b>Subject:</b>
RE:
[openstack-dev]
[nova]
[neutron] PCI
pass-through
network
support</span><span
style=""
lang="EN-US"></span></p>
</div>
</div>
<p
class="MsoNormal"><span
style=""
lang="EN-US"> </span></p>
<p
class="MsoNormal"><span
style="font-size:10.5pt;color:#1f497d" lang="EN-US">Robert, is it
possible to
have a IRC
meeting? I’d
prefer to IRC
meeting
because it’s
more openstack
style and also
can keep the
minutes
clearly. </span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="font-size:10.5pt;color:#1f497d" lang="EN-US"> </span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="font-size:10.5pt;color:#1f497d" lang="EN-US">To your flow, can
you give more
detailed
example. For
example, I can
consider user
specify the
instance with
–nic option
specify a
network id,
and then how
nova device
the
requirement to
the PCI
device? I
assume the
network id
should define
the switches
that the
device can
connect to ,
but how is
that
information
translated to
the PCI
property
requirement?
Will this
translation
happen before
the nova
scheduler make
host decision?</span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="font-size:10.5pt;color:#1f497d" lang="EN-US"> </span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="font-size:10.5pt;color:#1f497d" lang="EN-US">Thanks</span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="font-size:10.5pt;color:#1f497d" lang="EN-US">--jyh</span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="font-size:10.5pt;color:#1f497d" lang="EN-US"> </span><span
style=""
lang="EN-US"></span></p>
<div
style="border:none;border-left:solid
blue
1.5pt;padding:0cm
0cm 0cm 4.0pt">
<div>
<div
style="border:none;border-top:solid
#b5c4df
1.0pt;padding:3.0pt
0cm 0cm 0cm">
<p
class="MsoNormal"><b><span
style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">From:</span></b><span
style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">
Robert Li
(baoli) [<a
moz-do-not-send="true"
href="mailto:baoli@cisco.com" target="_blank">mailto:baoli@cisco.com</a>]
<br>
<b>Sent:</b>
Monday,
October 28,
2013 12:22 PM<br>
<b>To:</b>
Irena
Berezovsky; <a
moz-do-not-send="true" href="mailto:prashant.upadhyaya@aricent.com"
target="_blank">prashant.upadhyaya@aricent.com</a>;
Jiang,
Yunhong; <a
moz-do-not-send="true"
href="mailto:chris.friesen@windriver.com" target="_blank">chris.friesen@windriver.com</a>;
He, Yongli;
Itzik Brown<br>
<b>Cc:</b>
OpenStack
Development
Mailing List;
Brian Bowen
(brbowen);
Kyle Mestery
(kmestery);
Sandhya Dasu
(sadasu)<br>
<b>Subject:</b>
Re:
[openstack-dev]
[nova]
[neutron] PCI
pass-through
network
support</span><span
style=""
lang="EN-US"></span></p>
</div>
</div>
<p
class="MsoNormal"><span
style=""
lang="EN-US"> </span></p>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">Hi Irena,</span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">Thank you very much for your
comments. See
inline. </span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">--Robert</span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">On 10/27/13 3:48 AM, "Irena
Berezovsky"
<<a
moz-do-not-send="true"
href="mailto:irenab@mellanox.com" target="_blank">irenab@mellanox.com</a>>
wrote:</span><span
style=""
lang="EN-US"></span></p>
</div>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<blockquote
style="border:none;border-left:solid
#b5c4df
4.5pt;padding:0cm
0cm 0cm
4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">Hi Robert,</span><span style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">Thank you very much for sharing the
information
regarding your
efforts. Can
you please
share your
idea of the
end to end
flow? How do
you suggest
to bind Nova
and Neutron?</span><span
style=""
lang="EN-US"></span></p>
</div>
</div>
</blockquote>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">The end to end flow is actually
encompassed in
the blueprints
in a nutshell.
I will
reiterate it
in below. The
binding
between Nova
and Neutron
occurs with
the neutron v2
API that nova
invokes in
order to
provision the
neutron
services. The
vif driver is
responsible
for plugging
in an instance
onto the
networking
setup that
neutron has
created on the
host.</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">Normally, one will invoke "nova
boot" api with
the —nic
options to
specify the
nic with which
the instance
will be
connected to
the network.
It currently
allows net-id,
fixed ip
and/or port-id
to be
specified for
the option.
However, it
doesn't allow
one to specify
special
networking
requirements
for the
instance.
Thanks to the
nova
pci-passthrough
work, one can
specify PCI
passthrough
device(s) in
the nova
flavor. But it
doesn't
provide means
to tie up
these PCI
devices in the
case of
ethernet
adpators with
networking
services.
Therefore the
idea is
actually
simple as
indicated by
the blueprint
titles, to
provide means
to tie up
SRIOV devices
with neutron
services. A
work flow
would roughly
look like this
for 'nova
boot':</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> -- Specifies networking
requirements
in the —nic
option.
Specifically
for SRIOV,
allow the
following to
be specified
in addition to
the existing
required
information:</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> . PCI alias</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> . direct
pci-passthrough/macvtap</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> . port profileid
that is
compliant with
802.1Qbh</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> The above information is
optional. In
the absence of
them, the
existing
behavior
remains.</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> -- if special networking
requirements
exist, Nova
api creates
PCI requests
in the nova
instance type
for scheduling
purpose</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> -- Nova scheduler schedules
the instance
based on the
requested
flavor plus
the PCI
requests that
are created
for
networking.</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> -- Nova compute invokes
neutron
services with
PCI
passthrough
information if
any </span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> -- Neutron performs its
normal
operations
based on the
request, such
as allocating
a port,
assigning ip
addresses,
etc. Specific
to SRIOV, it
should
validate the
information
such as
profileid, and
stores them in
its db. It's
also possible
to associate a
port profileid
with a neutron
network so
that port
profileid
becomes
optional in
the —nic
option.
Neutron
returns nova
the port
information,
especially for
PCI
passthrough
related
information in
the port
binding
object.
Currently, the
port binding
object
contains the
following
information:</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> binding:vif_type</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> binding:host_id</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> binding:profile</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> binding:capabilities</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> -- nova constructs the domain
xml and plug
in the
instance by
calling the
vif driver.
The vif driver
can build up
the interface
xml based on
the port
binding
information. </span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<blockquote
style="border:none;border-left:solid
#b5c4df
4.5pt;padding:0cm
0cm 0cm
4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US"> </span><span style="" lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">The blueprints you registered make
sense. On Nova
side, there is
a need to bind
between
requested
virtual
network and
PCI
device/interface
to be
allocated as
vNIC.</span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">On the Neutron side, there is a need
to support
networking
configuration
of the vNIC.
Neutron should
be able to
identify the
PCI
device/macvtap
interface in
order to apply
configuration.
I think it
makes sense to
provide
neutron
integration
via dedicated
Modular Layer
2 Mechanism
Driver to
allow PCI
pass-through
vNIC support
along with
other
networking
technologies.</span><span
style=""
lang="EN-US"></span></p>
</div>
</div>
</blockquote>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">I haven't sorted through this yet.
A neutron port
could be
associated
with a PCI
device or not,
which is a
common
feature, IMHO.
However, a ML2
driver may be
needed
specific to a
particular
SRIOV
technology. </span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<blockquote
style="border:none;border-left:solid
#b5c4df
4.5pt;padding:0cm
0cm 0cm
4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US"> </span><span style="" lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">During the Havana Release, we
introduced
Mellanox
Neutron plugin
that enables
networking via
SRIOV
pass-through
devices or
macvtap
interfaces.</span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">We want to integrate our solution
with PCI
pass-through
Nova support.
I will be
glad to share
more details
if you are
interested.</span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US"> </span><span style="" lang="EN-US"></span></p>
</div>
</div>
</blockquote>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">Good to know that you already have
a SRIOV
implementation.
I found out
some
information
online about
the mlnx
plugin, but
need more time
to get to know
it better. And
certainly I'm
interested in
knowing its
details.</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<blockquote
style="border:none;border-left:solid
#b5c4df
4.5pt;padding:0cm
0cm 0cm
4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">The PCI pass-through networking
support is
planned to be
discussed
during the
summit: </span><span
style=""
lang="EN-US"><a
moz-do-not-send="true"
href="http://summit.openstack.org/cfp/details/129"
target="_blank">http://summit.openstack.org/cfp/details/129</a>. </span><span
style="color:#1f497d" lang="EN-US">I think it’s worth to drill down into
more detailed
proposal and
present it
during the
summit,
especially
since it
impacts both
nova and
neutron
projects. </span><span
style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US"> </span><span style="" lang="EN-US"></span></p>
</div>
</div>
</blockquote>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">I agree. Maybe we can steal some
time in that
discussion.</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<blockquote
style="border:none;border-left:solid
#b5c4df
4.5pt;padding:0cm
0cm 0cm
4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">Would you be interested in
collaboration
on this
effort? Would
you be
interested to
exchange more
emails or set
an IRC/WebEx
meeting during
this week
before the
summit?</span><span
style=""
lang="EN-US"></span></p>
</div>
</div>
</blockquote>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">Sure. If folks want to discuss it
before the
summit, we can
schedule a
webex later
this week. Or
otherwise, we
can continue
the discussion
with email.</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<blockquote
style="border:none;border-left:solid
#b5c4df
4.5pt;padding:0cm
0cm 0cm
4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US"> </span><span style="" lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US"> </span><span style="" lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">Regards,</span><span style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">Irena</span><span style=""
lang="EN-US"></span></p>
<p
class="MsoNormal"><span
style="color:#1f497d" lang="EN-US"> </span><span style="" lang="EN-US"></span></p>
<div>
<div
style="border:none;border-top:solid
#b5c4df
1.0pt;padding:3.0pt
0cm 0cm 0cm">
<p
class="MsoNormal"><b><span
style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">From:</span></b><span
style="font-size:10pt;font-family:Tahoma,sans-serif" lang="EN-US">
Robert Li
(baoli) [<a
moz-do-not-send="true"
href="mailto:baoli@cisco.com" target="_blank">mailto:baoli@cisco.com</a>]
<br>
<b>Sent:</b>
Friday,
October 25,
2013 11:16 PM<br>
<b>To:</b> <a
moz-do-not-send="true" href="mailto:prashant.upadhyaya@aricent.com"
target="_blank">prashant.upadhyaya@aricent.com</a>;
Irena
Berezovsky; <a
moz-do-not-send="true" href="mailto:yunhong.jiang@intel.com"
target="_blank">yunhong.jiang@intel.com</a>;
<a
moz-do-not-send="true"
href="mailto:chris.friesen@windriver.com" target="_blank">
chris.friesen@windriver.com</a>; <a moz-do-not-send="true"
href="mailto:yongli.he@intel.com"
target="_blank">yongli.he@intel.com</a><br>
<b>Cc:</b>
OpenStack
Development
Mailing List;
Brian Bowen
(brbowen);
Kyle Mestery
(kmestery);
Sandhya Dasu
(sadasu)<br>
<b>Subject:</b>
Re:
[openstack-dev]
[nova]
[neutron] PCI
pass-through
network
support</span><span
style=""
lang="EN-US"></span></p>
</div>
</div>
<p
class="MsoNormal"><span
style=""
lang="EN-US"> </span></p>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">Hi Irena,</span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">This is Robert Li from Cisco
Systems.
Recently, I
was tasked to
investigate
such support
for Cisco's
systems that
support
VM-FEX, which
is a SRIOV
technology
supporting
802-1Qbh. I
was able to
bring up nova
instances with
SRIOV
interfaces,
and establish
networking in
between the
instances that
employes the
SRIOV
interfaces.
Certainly,
this was
accomplished
with hacking
and some
manual
intervention.
Based on this
experience and
my study with
the two
existing nova
pci-passthrough
blueprints
that have been
implemented
and committed
into Havana (<a
moz-do-not-send="true"
href="https://blueprints.launchpad.net/nova/+spec/pci-passthrough-base"
target="_blank"><span
style="color:rgb(0,51,170);font-size:9pt;text-decoration:none;font-family:Arial,sans-serif;background-repeat:initial
initial">https://blueprints.launchpad.net/nova/+spec/pci-passthrough-base</span></a></span><span
style="color:rgb(51,51,51);font-size:9pt;font-family:Arial,sans-serif;background-repeat:initial
initial"
lang="EN-US"> and<br>
</span><span
style="color:rgb(0,51,170);font-size:9pt;font-family:Arial,sans-serif;background-repeat:initial
initial"
lang="EN-US"><a
moz-do-not-send="true"
href="https://blueprints.launchpad.net/nova/+spec/pci-passthrough-libvirt"
target="_blank"><span style="font-size:10.5pt">https://blueprints.launchpad.net/nova/+spec/pci-passthrough-libvirt</span></a>), </span><span
style="font-size:10.5pt" lang="EN-US"> I registered a couple of
blueprints
(one on Nova
side, the
other on the
Neutron side):</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"><a moz-do-not-send="true"
href="https://blueprints.launchpad.net/nova/+spec/pci-passthrough-sriov"
target="_blank">https://blueprints.launchpad.net/nova/+spec/pci-passthrough-sriov</a></span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"><a moz-do-not-send="true"
href="https://blueprints.launchpad.net/neutron/+spec/pci-passthrough-sriov"
target="_blank">https://blueprints.launchpad.net/neutron/+spec/pci-passthrough-sriov</a></span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">in order to address SRIOV support
in openstack. </span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">Please take a look at them and see
if they make
sense, and let
me know any
comments and
questions. We
can also
discuss this
in the summit,
I suppose.</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">I noticed that there is another
thread on this
topic, so copy
those folks
from that
thread as
well.</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">thanks,</span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">Robert</span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US">On 10/16/13 4:32 PM, "Irena
Berezovsky"
<<a
moz-do-not-send="true"
href="mailto:irenab@mellanox.com" target="_blank">irenab@mellanox.com</a>>
wrote:</span><span
style=""
lang="EN-US"></span></p>
</div>
<div>
<p
class="MsoNormal"><span
style="font-size:10.5pt" lang="EN-US"> </span><span style=""
lang="EN-US"></span></p>
</div>
<blockquote
style="border:none;border-left:solid
#b5c4df
4.5pt;padding:0cm
0cm 0cm
4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p
class="MsoNormal"><span
style=""
lang="EN-US">Hi,</span></p>
<p
class="MsoNormal"><span
style=""
lang="EN-US">As
one of the
next steps for
PCI
pass-through I
would like to
discuss is the
support for
PCI
pass-through
vNIC.</span></p>
<p
class="MsoNormal"><span
style=""
lang="EN-US">While
nova takes
care of PCI
pass-through
device
resources
management
and VIF
settings,
neutron should
manage their
networking
configuration.</span></p>
<p
class="MsoNormal"><span
style=""
lang="EN-US">I
would like to
register
asummit
proposal to
discuss the
support for
PCI
pass-through
networking.</span></p>
<p
class="MsoNormal"><span
style=""
lang="EN-US">I
am not sure
what would be
the right
topic to
discuss the
PCI
pass-through
networking,
since it
involve both
nova and
neutron. </span></p>
<p
class="MsoNormal"><span
style=""
lang="EN-US">There
is already a
session
registered by
Yongli on nova
topic to
discuss the
PCI
pass-through
next steps. </span></p>
<p
class="MsoNormal"><span
style=""
lang="EN-US">I
think PCI
pass-through
networking is
quite a big
topic and it
worth to have
a separate
discussion.</span></p>
<p
class="MsoNormal"><span
style=""
lang="EN-US">Is
there any
other people
who are
interested to
discuss it and
share their
thoughts and
experience?</span></p>
<p
class="MsoNormal"><span
style=""
lang="EN-US"> </span></p>
<p
class="MsoNormal"><span
style=""
lang="EN-US">Regards,</span></p>
<p
class="MsoNormal"><span
style=""
lang="EN-US">Irena</span></p>
<p
class="MsoNormal"><span
style=""
lang="EN-US"> </span></p>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><span
style="font-size:10.5pt"
lang="EN-US"> </span></p>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</span> </blockquote>
<br>
</div>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a moz-do-not-send="true"
href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a moz-do-not-send="true"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"
target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
OpenStack-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<br>
</body>
</html>