Open Stack

I have to use [yjiang5_1] prefix now :)

--jyh

From: Ian Wells [mailto:ijw.ubuntu at cack.org.uk]
Sent: Friday, January 10, 2014 3:55 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [nova] [neutron] PCI pass-through network support

On 11 January 2014 00:04, Jiang, Yunhong <yunhong.jiang at intel.com<mailto:yunhong.jiang at intel.com>> wrote:
[yjiang5] Really thanks for the summary and it is quite clear. So what's the object of "equivalent devices at host level"? Because 'equivalent device * to an end user *" is flavor, so is it 'equivalent to *scheduler*" or 'equivalent to *xxx*'? If equivalent to scheduler, then I'd take the pci_stats as a flexible group for scheduler

To the scheduler, indeed.  And with the group proposal the scheduler and end user equivalences are one and the same.
[yjiang5_1] Once use the proposal, then we missed the flexible for 'end user equivalences" and that's the reason I'm against the group :)


Secondly, for your definition of 'whitelist', I'm hesitate to your '*and*' because IMHO, 'and' means mixed two things together, otherwise, we can state in simply one sentence. For example, I prefer to have another configuration option to define the 'put devices in the group', or, if we extend it , be "define extra information like 'group name' for devices".

I'm not stating what we should do, or what the definitions should mean; I'm saying how they've been interpreted as weve discussed this in the past.  We've had issues in the past where we've had continuing difficulties in describing anything without coming back to a 'whitelist' (generally meaning 'matching expression, as an actual 'whitelist' is implied, rather than separately required, in a grouping system.
 Bearing in mind what you said about scheduling, and if we skip 'group' for a moment, then can I suggest (or possibly restate, because your comments are pointing in this direction):
- we allow extra information to be added at what is now the whitelisting stage, that just gets carried around with the device
[yjiang5] For 'added at ... whitelisting stage', see my above statement about the configuration. However, if you do want to use whitelist, I'm ok, but please keep in mind that it's two functionality combined: device you may assign *and* the group name for these devices.

Indeed - which is in fact what we've been proposing all along.


- when we're turning devices into flavors, we can also match on that extra information if we want (which means we can tag up the devices on the compute node if we like, according to taste, and then bundle them up by tag to make flavors; or we can add Neutron specific information and ignore it when making flavors)
[yjiang5] Agree. Currently we can only use vendor_id and device_id for flavor/alias, but we can extend it to cover such extra information since now it's a API.

- we would need to add a config param on the control host to decide which flags to group on when doing the stats (and they would additionally be the only params that would work for flavors, I think)
[yjiang5] Agree. And this is achievable because we switch the flavor to be API, then we can control the flavor creation process.

OK - so if this is good then I think the question is how we could change the 'pci_whitelist' parameter we have - which, as you say, should either *only* do whitelisting or be renamed - to allow us to add information.  Yongli has something along those lines but it's not flexible and it distinguishes poorly between which bits are extra information and which bits are matching expressions (and it's still called pci_whitelist) - but even with those criticisms it's very close to what we're talking about.  When we have that I think a lot of the rest of the arguments should simply resolve themselves.

[yjiang5_1] The reason that not easy to find a flexible/distinguishable change to pci_whitelist is because it combined two things. So a stupid/naive solution in my head is, change it to VERY generic name, 'pci_devices_information', and change schema as an array of {'devices_property'=regex exp, 'group_name' = 'g1'} dictionary, and the device_property expression can be 'address ==xxx, vendor_id == xxx' (i.e. similar with current white list),  and we can squeeze more into the "pci_devices_information" in future, like 'network_information' = xxx or "Neutron specific information" you required in previous mail. All keys other than 'device_property' becomes extra information, i.e. software defined property. These extra information will be carried with the PCI devices,. Some implementation details, A)we can limit the acceptable keys, like we only support 'group_name', 'network_id', or we can accept any keys other than reserved (vendor_id, device_id etc) one. B) if a device match 'device_property' in several entries, raise exception, or use the first one.

[yjiang5_1] Another thing need discussed is, as you pointed out, "we would need to add a config param on the control host to decide which flags to group on when doing the stats".  I agree with the design, but some details need decided. Where should it defined. If we a) define it in both control node and compute node, then it should be static defined (just change pool_keys in "/opt/stack/nova/nova/pci/pci_stats.py" to a configuration parameter) . Or b) define only in control node, then I assume the control node should be the scheduler node, and the scheduler manager need save such information, present a API to fetch such information and the compute node need fetch it on every update_available_resource() periodic task. I'd prefer to take a) option in first step. Your idea?

--
Ian.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140111/07d6ff82/attachment.html>