<div dir="ltr"><div>If there are N flavor types there are N match expressions so I think it's pretty much equivalent in terms of complexity. It looks like some sort of packing problem to me, trying to fit N objects into M boxes, hence my statement that it's not going to be easy, but that's just a gut feeling - some of the matches can be vague, such as only the vendor ID or a vendor and two device types, so it's not as simple as one flavor matching one stats row.<br>
-- <br></div>Ian.<br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On 13 January 2014 21:00, Jiang, Yunhong <span dir="ltr"><<a href="mailto:yunhong.jiang@intel.com" target="_blank">yunhong.jiang@intel.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div link="blue" vlink="purple" lang="ZH-CN">
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1f497d" lang="EN-US">Ian, not sure if I get your question. Why should scheduler get the number of flavor types requested? The scheduler will only translate the PCI
flavor to the pci property match requirement like it does now, (either vendor_id, device_id, or item in extra_info), then match the translated pci flavor, i.e. pci requests, to the pci stats.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1f497d" lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1f497d" lang="EN-US">Thanks<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1f497d" lang="EN-US">--jyh<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1f497d" lang="EN-US"><u></u> <u></u></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" lang="EN-US">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" lang="EN-US"> Ian Wells [mailto:<a href="mailto:ijw.ubuntu@cack.org.uk" target="_blank">ijw.ubuntu@cack.org.uk</a>]
<br>
<b>Sent:</b> Monday, January 13, 2014 10:57 AM</span></p><div><div class="h5"><br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<br>
<b>Subject:</b> Re: [openstack-dev] [nova] [neutron] PCI pass-through network support<u></u><u></u></div></div><p></p>
</div>
</div><div><div class="h5">
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US">It's worth noting that this makes the scheduling a computationally hard problem. The answer to that in this scheme is to reduce the number of inputs to trivialise the problem. It's going to be O(f(number of flavor types
requested, number of pci_stats pools)) and if you group appropriately there shouldn't be an excessive number of pci_stats pools. I am not going to stand up and say this makes it achievable - and if it doesn't them I'm not sure that anything would make overlapping
flavors achievable - but I think it gives us some hope.<br>
-- <u></u><u></u></span></p>
</div>
<p class="MsoNormal"><span lang="EN-US">Ian.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span lang="EN-US"><u></u> <u></u></span></p>
<div>
<p class="MsoNormal"><span lang="EN-US">On 13 January 2014 19:27, Jiang, Yunhong <<a href="mailto:yunhong.jiang@intel.com" target="_blank">yunhong.jiang@intel.com</a>> wrote:<u></u><u></u></span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1f497d" lang="EN-US">Hi, Robert, scheduler keep count based on pci_stats instead of the pci flavor.</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1f497d" lang="EN-US"> </span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1f497d" lang="EN-US">As stated by Ian at
<a href="https://www.mail-archive.com/openstack-dev@lists.openstack.org/msg13455.html" target="_blank">
https://www.mail-archive.com/openstack-dev@lists.openstack.org/msg13455.html</a> already, the flavor will only use the tags used by pci_stats.</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1f497d" lang="EN-US"> </span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1f497d" lang="EN-US">Thanks</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1f497d" lang="EN-US">--jyh</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1f497d" lang="EN-US"> </span><span lang="EN-US"><u></u><u></u></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" lang="EN-US">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" lang="EN-US"> Robert
Li (baoli) [mailto:<a href="mailto:baoli@cisco.com" target="_blank">baoli@cisco.com</a>]
<br>
<b>Sent:</b> Monday, January 13, 2014 8:22 AM</span><span lang="EN-US"><u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span lang="EN-US"><br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<br>
<b>Subject:</b> Re: [openstack-dev] [nova] [neutron] PCI pass-through network support<u></u><u></u></span></p>
</div>
</div>
</div>
<p class="MsoNormal"><span lang="EN-US"> <u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"" lang="EN-US">As I have responded in the other email, and If I understand PCI flavor correctly, then the issue that
we need to deal with is the overlapping issue. A simplest case of this overlapping is that you can define a flavor F1 as [vendor_id='v', product_id='p'], and a flavor F2 as [vendor_id = 'v'] . Let's assume that only the admin can define the flavors. It's
not hard to see that a device can belong to the two different flavors in the same time. This introduces an issue in the scheduler. Suppose the scheduler (counts or stats based) maintains counts based on flavors (or the keys corresponding to the flavors). To
request a device with the flavor F1, counts in F2 needs to be subtracted by one as well. There may be several ways to achieve that. But regardless, it introduces tremendous overhead in terms of system processing and administrative costs. </span><span lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"" lang="EN-US"> </span><span lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"" lang="EN-US">What are the use cases for that? How practical are those use cases?</span><span lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"" lang="EN-US"> </span><span lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"" lang="EN-US">thanks,</span><span lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"" lang="EN-US">Robert</span><span lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"" lang="EN-US"> </span><span lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"" lang="EN-US">On 1/10/14 9:34 PM, "Ian Wells" <<a href="mailto:ijw.ubuntu@cack.org.uk" target="_blank">ijw.ubuntu@cack.org.uk</a>>
wrote:</span><span lang="EN-US"><u></u><u></u></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"" lang="EN-US"> </span><span lang="EN-US"><u></u><u></u></span></p>
</div>
<blockquote style="border:none;border-left:solid #b5c4df 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif"" lang="EN-US"> <br>
><br>
> OK - so if this is good then I think the question is how we could change the 'pci_whitelist' parameter we have - which, as you say, should either *only* do whitelisting or be renamed - to allow us to add information. Yongli has something along those lines
but it's not flexible and it distinguishes poorly between which bits are extra information and which bits are matching expressions (and it's still called pci_whitelist) - but even with those criticisms it's very close to what we're talking about. When we
have that I think a lot of the rest of the arguments should simply resolve themselves.<br>
><br>
> <br>
><br>
> [yjiang5_1] The reason that not easy to find a flexible/distinguishable change to pci_whitelist is because it combined two things. So a stupid/naive solution in my head is, change it to VERY generic name, ‘pci_devices_information’,<br>
><br>
> and change schema as an array of {‘devices_property’=regex exp, ‘group_name’ = ‘g1’} dictionary, and the device_property expression can be ‘address ==xxx, vendor_id == xxx’ (i.e. similar with current white list), and we can squeeze more into the “pci_devices_information”
in future, like ‘network_information’ = xxx or “Neutron specific information” you required in previous mail.<br>
<br>
<br>
We're getting to the stage that an expression parser would be useful, annoyingly, but if we are going to try and squeeze it into JSON can I suggest:<br>
<br>
{ match = { class = "Acme inc. discombobulator" }, info = { group = "we like teh groups", volume = "11" } }<br>
<br>
><br>
> All keys other than ‘device_property’ becomes extra information, i.e. software defined property. These extra information will be carried with the PCI devices,. Some implementation details, A)we can limit the acceptable keys, like we only support ‘group_name’,
‘network_id’, or we can accept any keys other than reserved (vendor_id, device_id etc) one.<br>
<br>
<br>
Not sure we have a good list of reserved keys at the moment, and with two dicts it isn't really necessary, I guess. I would say that we have one match parser which looks something like this:<br>
<br>
# does this PCI device match the expression given?<br>
def match(expression, pci_details, extra_specs):<br>
for (k, v) in expression:<br>
if k.starts_with('e.'):<br>
mv = extra_specs.get(k[2:])<br>
else:<br>
mv = pci_details.get(k[2:])<br>
if not match(m, mv):<br>
return False<br>
return True<br>
<br>
Usable in this matching (where 'e.' just won't work) and also for flavor assignment (where e. will indeed match the extra values).<br>
<br>
> B) if a device match ‘device_property’ in several entries, raise exception, or use the first one.<br>
<br>
Use the first one, I think. It's easier, and potentially more useful.<br>
<br>
> [yjiang5_1] Another thing need discussed is, as you pointed out, “we would need to add a config param on the control host to decide which flags to group on when doing the stats”. I agree with the design, but some details need decided.
<br>
<br>
This is a patch that can come at any point after we do the above stuff (which we need for Neutron), clearly.<br>
<br>
> Where should it defined. If we a) define it in both control node and compute node, then it should be static defined (just change pool_keys in "/opt/stack/nova/nova/pci/pci_stats.py" to a configuration parameter) . Or b) define only in control node, then I
assume the control node should be the scheduler node, and the scheduler manager need save such information, present a API to fetch such information and the compute node need fetch it on every update_available_resource() periodic task. I’d prefer to take a)
option in first step. Your idea?<br>
<br>
I think it has to be (a), which is a shame.</span><span lang="EN-US"><u></u><u></u></span></p>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span lang="EN-US"><br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><u></u><u></u></span></p>
</div>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
</div>
</div></div></div>
</div>
</div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>