<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
@font-face
{font-family:"\@SimSun";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Times New Roman","serif";}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:SimSun;}
span.EmailStyle19
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-CA" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">+1 PCI Flavor.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="margin-left:36.0pt"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Jiang, Yunhong [mailto:yunhong.jiang@intel.com]
<br>
<b>Sent:</b> January-10-14 1:56 AM<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<br>
<b>Subject:</b> Re: [openstack-dev] [nova] [neutron] PCI pass-through network support<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="margin-left:36.0pt"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN">BTW, I like the PCI flavor
</span><span lang="EN-US" style="font-size:10.5pt;font-family:Wingdings;color:#1F497D;mso-fareast-language:ZH-CN">J</span><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="margin-left:36.0pt"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif";mso-fareast-language:ZH-CN">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif";mso-fareast-language:ZH-CN">
Jiang, Yunhong [<a href="mailto:yunhong.jiang@intel.com">mailto:yunhong.jiang@intel.com</a>]
<br>
<b>Sent:</b> Thursday, January 09, 2014 10:41 PM<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<br>
<b>Subject:</b> Re: [openstack-dev] [nova] [neutron] PCI pass-through network support<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN">Hi, Ian, when you in aggrement with all of this, do you agree with the ‘group name’, or agree
with John’s pci flavor? <o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN">I’m against the PCI group and will send out a reply later.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN">--jyh<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="margin-left:36.0pt"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif";mso-fareast-language:ZH-CN">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif";mso-fareast-language:ZH-CN">
Ian Wells [<a href="mailto:ijw.ubuntu@cack.org.uk">mailto:ijw.ubuntu@cack.org.uk</a>]
<br>
<b>Sent:</b> Thursday, January 09, 2014 9:47 AM<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<br>
<b>Subject:</b> Re: [openstack-dev] [nova] [neutron] PCI pass-through network support<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:12.0pt;margin-left:36.0pt">
<span lang="EN-US" style="mso-fareast-language:ZH-CN">I think I'm in agreement with all of this. Nice summary, Robert.<o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="mso-fareast-language:ZH-CN">It may not be where the work ends, but if we could get this done the rest is just refinement.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:12.0pt;margin-left:36.0pt">
<span lang="EN-US" style="mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="mso-fareast-language:ZH-CN">On 9 January 2014 17:49, Robert Li (baoli) <<a href="mailto:baoli@cisco.com" target="_blank">baoli@cisco.com</a>> wrote:<o:p></o:p></span></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:36.0pt">
<span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN">Hi Folks,</span><span lang="EN-US" style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN">With John joining the IRC, so far, we had a couple of productive meetings in an effort to come to consensus
and move forward. Thanks John for doing that, and I appreciate everyone's effort to make it to the daily meeting. Let's reconvene on Monday. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN">But before that, and based on our today's conversation on IRC, I'd like to say a few things. I think that
first of all, we need to get agreement on the terminologies that we are using so far. With the current nova PCI passthrough<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"> PCI whitelist: defines all the available PCI passthrough devices on a compute node. </span><span lang="EN-US" style="font-size:10.0pt;font-family:Consolas;color:#333333;background:whitesmoke;mso-fareast-language:ZH-CN">pci_passthrough_whitelist=[{
"vendor_id":"xxxx","product_id":"xxxx"}] </span><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"> PCI Alias: criteria defined on the controller node with which requested PCI passthrough devices
can be selected from all the PCI passthrough devices available in a cloud. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"> Currently it has the following format: </span><span lang="EN-US" style="font-size:10.0pt;font-family:Consolas;color:#333333;background:whitesmoke;mso-fareast-language:ZH-CN">pci_alias={"vendor_id":"xxxx",
"product_id":"xxxx", "name":"str"}</span><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"> nova flavor extra_specs: request for PCI passthrough devices can be specified with extra_specs
in the format for example:</span><span lang="EN-US" style="font-size:10.0pt;font-family:Consolas;color:#333333;background:whitesmoke;mso-fareast-language:ZH-CN">"pci_passthrough:alias"="name:count"</span><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN">As you can see, currently a PCI alias has a name and is defined on the controller. The implications for
it is that when matching it against the PCI devices, it has to match the vendor_id and product_id against all the available PCI devices until one is found. The name is only used for reference in the extra_specs. On the other hand, the whitelist is basically
the same as the alias without a name.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN">What we have discussed so far is based on something called PCI groups (or PCI flavors as Yongli puts it).
Without introducing other complexities, and with a little change of the above representation, we will have something like:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.0pt;font-family:Consolas;color:#333333;background:whitesmoke;mso-fareast-language:ZH-CN">pci_passthrough_whitelist=[{ "vendor_id":"xxxx","product_id":"xxxx", "name":"str"}] </span><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN">By doing so, we eliminated the PCI alias. And we call the "name" in above as a PCI group name. You can
think of it as combining the definitions of the existing whitelist and PCI alias. And believe it or not, a PCI group is actually a PCI alias. However, with that change of thinking, a lot of benefits can be harvested:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"> * the implementation is significantly simplified<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"> * provisioning is simplified by eliminating the PCI alias<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"> * a compute node only needs to report stats with something like: PCI group name:count. A compute
node processes all the PCI passthrough devices against the whitelist, and assign a PCI group based on the whitelist definition.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"> * on the controller, we may only need to define the PCI group names. if we use a nova api to define
PCI groups (could be private or public, for example), one potential benefit, among other things (validation, etc), they can be owned by the tenant that creates them. And thus a wholesale of PCI passthrough devices is also possible.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"> * scheduler only works with PCI group names. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"> * request for PCI passthrough device is based on PCI-group<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"> * deployers can provision the cloud based on the PCI groups<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"> * Particularly for SRIOV, deployers can design SRIOV PCI groups based on network connectivities.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN">Further, to support SRIOV, we are saying that PCI group names not only can be used in the extra specs,
it can also be used in the —nic option and the neutron commands. This allows the most flexibilities and functionalities afforded by SRIOV.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN">Further, we are saying that we can define default PCI groups based on the PCI device's class.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN">For vnic-type (or nic-type), we are saying that it defines the link characteristics of the nic that is
attached to a VM: a nic that's connected to a virtual switch, a nic that is connected to a physical switch, or a nic that is connected to a physical switch, but has a host macvtap device in between. The actual names of the choices are not important here, and
can be debated.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN">I'm hoping that we can go over the above on Monday. But any comments are welcome by email.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN">Thanks,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN">Robert<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri","sans-serif";mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:12.0pt;margin-left:36.0pt">
<span lang="EN-US" style="mso-fareast-language:ZH-CN"><br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US" style="mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
</div>
</div>
</div>
</div>
</body>
</html>