<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.EmailStyle17
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:376590131;
mso-list-type:hybrid;
mso-list-template-ids:1838349382 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l1
{mso-list-id:1336765813;
mso-list-type:hybrid;
mso-list-template-ids:82591696 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l1:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l1:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l1:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l2
{mso-list-id:1982805414;
mso-list-type:hybrid;
mso-list-template-ids:-68499024 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l2:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l2:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l2:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hi,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">After having a lot of discussions both on IRC and mailing list, I would like to suggest to define basic use cases for PCI pass-through network support with
agreed list of limitations and assumptions and implement it. By doing this Proof of Concept we will be able to deliver basic PCI pass-through network support in Icehouse timeframe and understand better how to provide complete solution starting from tenant
/admin API enhancement, enhancing nova-neutron communication and eventually provide neutron plugin supporting the PCI pass-through networking.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">We can try to split tasks between currently involved participants and bring up the basic case. Then we can enhance the implementation.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Having more knowledge and experience with neutron parts, I would like to start working on neutron mechanism driver support. I have already started to arrange
the following blueprint doc based on everyone’s ideas:<o:p></o:p></span></p>
<p class="MsoNormal"><a href="https://docs.google.com/document/d/1RfxfXBNB0mD_kH9SamwqPI8ZM-jg797ky_Fze7SakRc/edit">https://docs.google.com/document/d/1RfxfXBNB0mD_kH9SamwqPI8ZM-jg797ky_Fze7SakRc/edit#</a><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">For the basic PCI pass-through networking case we can assume the following:<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l2 level1 lfo3"><![if !supportLists]><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span dir="LTR"></span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Single provider network (PN1)<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l2 level1 lfo3"><![if !supportLists]><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><span style="mso-list:Ignore">2.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span dir="LTR"></span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">White list of available SRIOV PCI devices for allocation as NIC for neutron networks on provider network (PN1) is defined
on each compute node <o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l2 level1 lfo3"><![if !supportLists]><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><span style="mso-list:Ignore">3.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span dir="LTR"></span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Support directly assigned SRIOV PCI pass-through device as vNIC. (This will limit the number of tests)<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l2 level1 lfo3"><![if !supportLists]><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><span style="mso-list:Ignore">4.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span dir="LTR"></span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">More ….<o:p></o:p></span></p>
<p class="MsoListParagraph"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">If my suggestion seems reasonable to you, let’s try to reach an agreement and split the work during our Monday IRC meeting.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">BR,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Irena<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Jiang, Yunhong [mailto:yunhong.jiang@intel.com]
<br>
<b>Sent:</b> Saturday, January 11, 2014 8:36 AM<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<br>
<b>Subject:</b> Re: [openstack-dev] [nova] [neutron] PCI pass-through network support<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN">Comments with prefix [yjiang5_2] , including the double confirm.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN">I think we (you and me) is mostly on the same page, would you please give a summary, and then we can have community , including Irena/Robert,
to check it. We need Cores to sponsor it. We should check with John to see if this is different with his mentor picture, and we may need a neutron core (I assume Cisco has a bunch of Neutron cores
</span><span style="font-size:10.5pt;font-family:Wingdings;color:#1F497D;mso-fareast-language:ZH-CN">J</span><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN"> )to sponsor it?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN">And, will anyone from Cisco can help on the implementation? After this long discussion, we are in half bottom of I release and I’m
not sure if Yongli and I alone can finish them in I release.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN">Thanks<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN">--jyh<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";mso-fareast-language:ZH-CN">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";mso-fareast-language:ZH-CN"> Ian Wells [<a href="mailto:ijw.ubuntu@cack.org.uk">mailto:ijw.ubuntu@cack.org.uk</a>]
<br>
<b>Sent:</b> Friday, January 10, 2014 6:34 PM<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<br>
<b>Subject:</b> Re: [openstack-dev] [nova] [neutron] PCI pass-through network support<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span style="mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span style="mso-fareast-language:ZH-CN"> <br>
><br>
> OK - so if this is good then I think the question is how we could change the 'pci_whitelist' parameter we have - which, as you say, should either *only* do whitelisting or be renamed - to allow us to add information. Yongli has something along those lines
but it's not flexible and it distinguishes poorly between which bits are extra information and which bits are matching expressions (and it's still called pci_whitelist) - but even with those criticisms it's very close to what we're talking about. When we
have that I think a lot of the rest of the arguments should simply resolve themselves.<br>
><br>
> <br>
><br>
> [yjiang5_1] The reason that not easy to find a flexible/distinguishable change to pci_whitelist is because it combined two things. So a stupid/naive solution in my head is, change it to VERY generic name, ‘pci_devices_information’,<br>
><br>
> and change schema as an array of {‘devices_property’=regex exp, ‘group_name’ = ‘g1’} dictionary, and the device_property expression can be ‘address ==xxx, vendor_id == xxx’ (i.e. similar with current white list), and we can squeeze more into the “pci_devices_information”
in future, like ‘network_information’ = xxx or “Neutron specific information” you required in previous mail.<br>
<br>
<br>
We're getting to the stage that an expression parser would be useful, annoyingly, but if we are going to try and squeeze it into JSON can I suggest:<br>
<br>
{ match = { class = "Acme inc. discombobulator" }, info = { group = "we like teh groups", volume = "11" } }<br>
<span style="color:#1F497D"><o:p></o:p></span></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN">[yjiang5_2] Double confirm that ‘match’ is whitelist, and info is ‘extra info’, right? Can the key be more meaningful, for example,
s/match/pci_device_property, s/info/pci_device_info, or s/match/pci_devices/ etc.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN">Also assume the class should be the class code in the configuration space, and be digital, am I right? Otherwise, it’s not easy to
get the ‘Acme inc. discombobulator’ information.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:ZH-CN"><br>
><br>
> All keys other than ‘device_property’ becomes extra information, i.e. software defined property. These extra information will be carried with the PCI devices,. Some implementation details, A)we can limit the acceptable keys, like we only support ‘group_name’,
‘network_id’, or we can accept any keys other than reserved (vendor_id, device_id etc) one.<br>
<br>
<br>
Not sure we have a good list of reserved keys at the moment, and with two dicts it isn't really necessary, I guess. I would say that we have one match parser which looks something like this:<br>
<br>
# does this PCI device match the expression given?<br>
def match(expression, pci_details, extra_specs):<br>
for (k, v) in expression:<br>
if k.starts_with('e.'):<br>
mv = extra_specs.get(k[2:])<br>
else:<br>
mv = pci_details.get(k[2:])<br>
if not match(m, mv):<br>
return False<br>
return True<br>
<br>
Usable in this matching (where 'e.' just won't work) and also for flavor assignment (where e. will indeed match the extra values).<span style="color:#1F497D"><o:p></o:p></span></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN">[yjiang5_2] I think if we use same function to check or use two functions for match/flavor will be implementation detail and can
be discussed in next step. Of course, we should always avoid code duplication.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:ZH-CN"><br>
<br>
> B) if a device match ‘device_property’ in several entries, raise exception, or use the first one.<br>
<br>
Use the first one, I think. It's easier, and potentially more useful.<span style="color:#1F497D"><o:p></o:p></span></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN">[yjiang5] good.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:ZH-CN"><br>
<br>
> [yjiang5_1] Another thing need discussed is, as you pointed out, “we would need to add a config param on the control host to decide which flags to group on when doing the stats”. I agree with the design, but some details need decided.
<br>
<br>
This is a patch that can come at any point after we do the above stuff (which we need for Neutron), clearly.<br>
<br>
> Where should it defined. If we a) define it in both control node and compute node, then it should be static defined (just change pool_keys in "/opt/stack/nova/nova/pci/pci_stats.py" to a configuration parameter) . Or b) define only in control node, then I
assume the control node should be the scheduler node, and the scheduler manager need save such information, present a API to fetch such information and the compute node need fetch it on every update_available_resource() periodic task. I’d prefer to take a)
option in first step. Your idea?<br>
<br>
I think it has to be (a), which is a shame.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN">[yjiang5] We can extend it to (b) in future.
<o:p></o:p></span></p>
</div>
</div>
</div>
</body>
</html>