<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">Ian,<div><br></div><div>The idea of pci flavors is a great and using vendor_id and product_id make sense, but I could see a case for adding the class name such as 'VGA compatible controller'. Otherwise, slightly different generations of hardware will mean custom whitelist setups on each compute node. </div><div><br></div><div>01:00.0 VGA compatible controller: NVIDIA Corporation G71 [GeForce 7900 GTX] (rev a1)<br></div><div><div><br></div></div><div>On the flip side, vendor_id and product_id might not be sufficient. Suppose I have two identical NICs, one for nova internal use and the second for guest tenants? So, bus numbering may be required. </div><div><br></div><div><div>01:00.0 VGA compatible controller: NVIDIA Corporation G71 [GeForce 7900 GTX] (rev a1)<br></div><div></div><div>02:00.0 VGA compatible controller: NVIDIA Corporation G71 [GeForce 7900 GTX] (rev a1)<br></div></div><div><br></div><div>Some possible combinations:</div><div><br></div><div><div># take 2 gpus</div><div>pci_passthrough_whitelist=[</div><div><span class="Apple-tab-span" style="white-space: pre;"> </span>{ "vendor_id":"NVIDIA Corporation G71","product_id":"GeForce 7900 GTX", "name":"GPU"},</div><div>]</div></div><div><br></div><div><div><div># only take the GPU on PCI 2</div><div>pci_passthrough_whitelist=[</div><div><span class="Apple-tab-span" style="white-space: pre;"> </span>{ "vendor_id":"NVIDIA Corporation G71","product_id":"GeForce 7900 GTX", 'bus_id': '02:', "name":"GPU"},</div><div>]</div></div><div>pci_passthrough_whitelist=[</div><div><span class="Apple-tab-span" style="white-space: pre;"> </span>{"bus_id": "01:00.0", "name": "GPU"},</div><div><div><span class="Apple-tab-span" style="white-space: pre;"> </span>{"bus_id": "02:00.0", "name": "GPU"},</div></div><div>]</div></div><div><br></div><div><div>pci_passthrough_whitelist=[</div><div><span class="Apple-tab-span" style="white-space: pre;"> </span>{"class": "VGA compatible controller", "name": "GPU"},</div><div>]</div></div><div><br></div><div><div><div><div><div>pci_passthrough_whitelist=[</div><div><span class="Apple-tab-span" style="white-space: pre;"> </span>{ "product_id":"GeForce 7900 GTX", "name":"GPU"},</div><div>]</div></div></div></div></div><div><br></div><div><div><div>I know you guys are thinking of PCI devices, but any though of mapping to something like udev rather than pci? Supporting udev rules might be easier and more robust rather than making something up.</div></div></div><div><br></div><div>Brian</div><div><br></div><div><div apple-content-edited="true">
<span class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px;"><div><div style="font-size: 12px; ">-------------------------------------------------</div><div style="font-size: 12px; ">Brian Schott, CTO</div><div style="font-size: 12px; ">Nimbis Services, Inc.</div><div style="font-size: 12px; "><a href="mailto:brian.schott@nimbisservices.com">brian.schott@nimbisservices.com</a></div><div style="font-size: 12px; ">ph: 443-274-6064 fx: 443-274-6060</div></div><div><br></div></span><br class="Apple-interchange-newline">
</div>
<br><div><div>On Jan 9, 2014, at 12:47 PM, Ian Wells <<a href="mailto:ijw.ubuntu@cack.org.uk">ijw.ubuntu@cack.org.uk</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div dir="ltr"><div>I think I'm in agreement with all of this. Nice summary, Robert.<br><br></div>It may not be where the work ends, but if we could get this done the rest is just refinement.<br></div><div class="gmail_extra">
<br><br><div class="gmail_quote">On 9 January 2014 17:49, Robert Li (baoli) <span dir="ltr"><<a href="mailto:baoli@cisco.com" target="_blank">baoli@cisco.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex; position: static; z-index: auto;">
<div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word">
<span>
<div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word">
<span>
<div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word">
<span>
<div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word">
<span>
<div style="word-wrap:break-word">
<span style="font-size:14px;font-family:Calibri,sans-serif">
<div>
<div link="blue" vlink="purple" lang="EN-US">
<div><p class="MsoNormal"><span style="font-family:Calibri,sans-serif;font-size:14px">Hi Folks,</span></p>
</div>
</div>
</div>
</span></div>
</span></div>
</span></div>
</span></div>
</span>
<div><br>
</div>
<div>With John joining the IRC, so far, we had a couple of productive meetings in an effort to come to consensus and move forward. Thanks John for doing that, and I appreciate everyone's effort to make it to the daily meeting. Let's reconvene on Monday. </div>
<div><br>
</div>
<div>But before that, and based on our today's conversation on IRC, I'd like to say a few things. I think that first of all, we need to get agreement on the terminologies that we are using so far. With the current nova PCI passthrough</div>
<div><br>
</div>
<div> PCI whitelist: defines all the available PCI passthrough devices on a compute node. <span style="background-color:rgb(245,245,245);color:rgb(51,51,51);font-family:Monaco,Menlo,Consolas,'Courier New',monospace;font-size:13px;line-height:20px;white-space:pre-wrap">pci_passthrough_whitelist=[{
"vendor_id":"xxxx","product_id":"xxxx"}] </span></div>
<div> PCI Alias: criteria defined on the controller node with which requested PCI passthrough devices can be selected from all the PCI passthrough devices available in a cloud. </div>
<div> Currently it has the following format: <span style="background-color:rgb(245,245,245);color:rgb(51,51,51);font-family:Monaco,Menlo,Consolas,'Courier New',monospace;font-size:13px;line-height:20px;white-space:pre-wrap">pci_alias={"vendor_id":"xxxx",
"product_id":"xxxx", "name":"str"}</span></div>
<div> </div>
<div> nova flavor extra_specs: request for PCI passthrough devices can be specified with extra_specs in the format for example:<span style="background-color:rgb(245,245,245);color:rgb(51,51,51);font-family:Monaco,Menlo,Consolas,'Courier New',monospace;font-size:13px;line-height:20px;white-space:pre-wrap">"pci_passthrough:alias"="name:count"</span></div>
<div><br>
</div>
<div>As you can see, currently a PCI alias has a name and is defined on the controller. The implications for it is that when matching it against the PCI devices, it has to match the vendor_id and product_id against all the available PCI devices until one is
found. The name is only used for reference in the extra_specs. On the other hand, the whitelist is basically the same as the alias without a name.</div>
<div><br>
</div>
<div>What we have discussed so far is based on something called PCI groups (or PCI flavors as Yongli puts it). Without introducing other complexities, and with a little change of the above representation, we will have something like:</div>
<div> </div>
<div><span style="color:rgb(51,51,51);font-family:Monaco,Menlo,Consolas,'Courier New',monospace;font-size:13px;line-height:20px;white-space:pre-wrap;background-color:rgb(245,245,245)">pci_passthrough_whitelist=[{ "vendor_id":"xxxx","product_id":"xxxx",
"name":"str"}] </span></div>
<div><br>
</div>
<div>By doing so, we eliminated the PCI alias. And we call the "name" in above as a PCI group name. You can think of it as combining the definitions of the existing whitelist and PCI alias. And believe it or not, a PCI group is actually a PCI alias. However,
with that change of thinking, a lot of benefits can be harvested:</div>
<div><br>
</div>
<div> * the implementation is significantly simplified</div>
<div> * provisioning is simplified by eliminating the PCI alias</div>
<div> * a compute node only needs to report stats with something like: PCI group name:count. A compute node processes all the PCI passthrough devices against the whitelist, and assign a PCI group based on the whitelist definition.</div>
<div> * on the controller, we may only need to define the PCI group names. if we use a nova api to define PCI groups (could be private or public, for example), one potential benefit, among other things (validation, etc), they can be owned by the tenant
that creates them. And thus a wholesale of PCI passthrough devices is also possible.</div>
<div> * scheduler only works with PCI group names. </div>
<div> * request for PCI passthrough device is based on PCI-group</div>
<div> * deployers can provision the cloud based on the PCI groups</div>
<div> * Particularly for SRIOV, deployers can design SRIOV PCI groups based on network connectivities.</div>
<div><br>
</div>
<div>Further, to support SRIOV, we are saying that PCI group names not only can be used in the extra specs, it can also be used in the —nic option and the neutron commands. This allows the most flexibilities and functionalities afforded by SRIOV.</div>
<div><br>
</div>
<div>Further, we are saying that we can define default PCI groups based on the PCI device's class.</div>
<div><br>
</div>
<div>For vnic-type (or nic-type), we are saying that it defines the link characteristics of the nic that is attached to a VM: a nic that's connected to a virtual switch, a nic that is connected to a physical switch, or a nic that is connected to a physical
switch, but has a host macvtap device in between. The actual names of the choices are not important here, and can be debated.</div>
<div><br>
</div>
<div>I'm hoping that we can go over the above on Monday. But any comments are welcome by email.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Robert</div>
<div><br>
</div>
</div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>
_______________________________________________<br>OpenStack-dev mailing list<br><a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev<br></blockquote></div><br></div></body></html>