HTML: https://anticdent.org/placement-update-19-09.html Here's another placement update. It was feature freeze this week so people are probably a bit worn, I'll try to keep this light and quick. # Most Important We need to decide if we want to do what amounts to a feature freeze exception for Tetsuro's negative member-of handling linked below and work out how/when to fit Mel's allocation ratio work on osc-placement (also below) into the world so people can use it. If people have placement related forum sessions they would like to see happen the deadline is either today (March 8th) or Monday, depending on which and how you read various pieces of info. Matt has already set up an extraction-related session. It's also important to be thinking about how placement would like to engage (as a group) with the PTG. If you want to run for PTL (of placement or anything else) the deadline is 23:45 UTC, 12 March. # What's Changed * The refactoring of the objects/resource_provider.py file into smaller, more single-purpose files, continues to merge. In case anyone is feeling like this is a lot of churn for little purpose, it does have purpose: This is making the code more accessible to people who aren't familiar with it. It is also providing a good review (a variety of warts and bugs and have been found and fixed) of stale code. * pep8's whitespace handling has been turned back on in placement. We inherited the exceptions to the rules from nova, where fixing for those rules would have been messy. In placement it wasn't that much of a big deal. * A conf setting `[placement_database]/sync_on_startup` has been added. Defaults to `False`. If `True`, the web service process will attempt the equivalent of `placement-manage db sync` at startup. * Improved debug logging when requesting allocation candidates, so that operators can more accurately determine what requirement led to reduced or no available hosts. * [1.5.0 of osc-placement](https://pypi.org/p/osc-placement) was released. It now goes up to microversion 1.18 (filter resource providers by required traits). (Note to everyone: We should make the info that shows up on PyPI more useful, by changing the README). * VGPU reshaping for libvirt merged! # Specs/Blueprints/Features ## Near to Done * [Filter Allocation Candidates by Provider Tree](http://specs.openstack.org/openstack/nova-specs/specs/stein/approved/alloc-c...) has been mostly completed by Tetsuro, but there's a [pending update to the spec](https://review.openstack.org/639033). * [Support filtering by forbidden aggregate membership](http://specs.openstack.org/openstack/nova-specs/specs/stein/approved/negativ...) also mostly [completed by Tetsuro](https://review.openstack.org/#/q/topic:bp/negative-aggregate-membership). ## Not yet Done These will get punted to Train. * [Support any traits in allocation_candidates query](http://specs.openstack.org/openstack/nova-specs/specs/stein/approved/placeme...) * [Support mixing required traits with any traits](http://specs.openstack.org/openstack/nova-specs/specs/stein/approved/placeme...) ## Not yet Approved * [Resource provider - request group mapping in allocation candidate](https://review.openstack.org/#/c/597601/) has had a recent resurgence in attention. # Bugs There is a storyboard [project group](https://storyboard.openstack.org/#!/project_group/placement) for placement projects. Through to the end of Stein we'll be paying attention to both it and Launchpad. The sole story in storyboard right now is updating docs to indicate we are using storyboard. So meta! In launchpad: * Placement related [bugs not yet in progress](https://goo.gl/TgiPXb):13. -2. * [In progress placement bugs](https://goo.gl/vzGGDQ) 9. -8. That large drop on in progress is because I went through and caught up those bugs which had fixes committed but had not been automatically updated. # osc-placement osc-placement is currently behind by 13 microversions. Pending changes: * [support for 1.19](https://review.openstack.org/#/c/641094/) * [support for 1.21](https://review.openstack.org/#/c/641123/) * [aggregate allocation ratio tool](https://review.openstack.org/#/c/640898/) # Main Themes ## Nested * VGPU reshaping for libvirt merged! * The bandwidth-resource-provider topic has merged a vast amount of code but there is [some left](https://review.openstack.org/#/q/topic:bp/bandwidth-resource-provider). Some of it may still merge in Stein, but the rest will be for Train. The microversion ([2.72](https://docs.openstack.org/nova/latest/reference/api-microversion-history.ht...) provides the API-level bits to allow booting a host with a minimum bandwidth requirement. At least some of the [remaining changes](https://review.openstack.org/#/c/637953) will be backported. There were no nibbles on last week's plea, so I'll say again: If anyone reading this is in a position to provide third party CI with fancy hardware for NUMA, NFV, FPGA, and GPU related integration testing with nova, there's a significant need for that. ## Refactoring The work that removed oslo.versionedobjects then moved on to removing the List classes (e.g. AllocationList) in favor of using native Python lists and breaking up the `resource_provider.py` file into smaller files. The first portion of that (`scrub-Lists`) merged. The next stage is on [cd/de-list-phase2](https://review.openstack.org/#/q/topic:cd/de-list-phase2). This will continue until each type has its own file. # Other Placement * <https://review.openstack.org/#/c/639628/> Docs: extract testing info to own sub-page * <https://review.openstack.org/#/q/topic:cd/gabbi-tempest-job> Gabbi-based integration tests of placement. These recently found a bug that none of the functional, grenade, nor tempest tests did. * <https://review.openstack.org/#/c/630216/> Add a vision-reflection (of the Technical Vision doc). * <https://review.openstack.org/#/q/topic:bp/negative-aggregate-membership> Negative member of aggregate filtering resource providers and allocation candidates. * <https://review.openstack.org/#/c/641422/> Docs: Group API versions by release * <https://review.openstack.org/#/c/640555/> Add explicit short-circuit in get_all_by_consumer_id # Other Service Users (If you stick "placement" somewhere in your commit message I'll probably eventually find your in-progress placement-related changes.) ## Nova * <https://review.openstack.org/#/q/topic:bp/count-quota-usage-from-placement> Using placement (from nova) for counting (some of) quota. Some of this, mostly related to database migrations, managed to make it in before feature freeze, but not the part that actually uses placement to do some counting, that is still pending. * <https://review.openstack.org/621494> Add descriptions of numbered resource classes and traits ## Not Nova * <https://review.openstack.org/#/q/topic:tripleo-nova-placement-removal> * <https://review.openstack.org/#/q/topic:tripleo-placement-extraction> * <https://review.openstack.org/#/q/topic:minimum-bandwidth-allocation-placement-api> Neutron side of minimum bandwidth. * <https://review.openstack.org/#/q/bp/no-affinity-instance-reservation> Blazar reservation handling, including some manipulation of inventory in placement. * <https://review.openstack.org/633204> Blazar: Retry on inventory update conflict * <https://review.openstack.org/#/c/626382/> Kolla: Copy placement database migration script * <https://review.openstack.org/#/c/629253/> Tempest: Minimum bandwidth allocation with placement # End Go forth and document and bug fix and raise a full and flavorful stein. -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent tw: @anticdent
On 3/8/2019 8:51 AM, Chris Dent wrote:
We need to decide if we want to do what amounts to a feature freeze exception for Tetsuro's negative member-of handling linked below and work out how/when to fit Mel's allocation ratio work on osc-placement (also below) into the world so people can use it.
Probably not surprisingly I'm a -1 on pushing this stuff in after feature freeze. We have two weeks to RC1 and people should be focusing on testing, bug triage, release notes and other feature documentation. Also, I'm pretty sure there is nothing in either nova or blazar that is currently not going to merge in stein if this doesn't land in Stein, so based on that it can wait until Train IMO. As for the osc-placement change, I think we can also defer that to Train. It's been talked about since the Dublin PTG and is just now getting implemented, which is good, but also means it's apparently low priority for people and therefore can wait until Train (we won't have another osc-placement release in Stein anyway since clients are frozen now). Also, since it's a client, it will work with older versions of the placement API anyway, i.e. osc-placement in Train should work fine with placement from Queens, so again I don't see a rush here.
There were no nibbles on last week's plea, so I'll say again: If anyone reading this is in a position to provide third party CI with fancy hardware for NUMA, NFV, FPGA, and GPU related integration testing with nova, there's a significant need for that.
There is some discussion happening in OpenLab about this: https://github.com/theopenlab/openlab/issues/200 -- Thanks, Matt
On Fri, 8 Mar 2019, Matt Riedemann wrote:
On 3/8/2019 8:51 AM, Chris Dent wrote:
We need to decide if we want to do what amounts to a feature freeze exception for Tetsuro's negative member-of handling linked below and work out how/when to fit Mel's allocation ratio work on osc-placement (also below) into the world so people can use it.
Probably not surprisingly I'm a -1 on pushing this stuff in after feature freeze. We have two weeks to RC1 and people should be focusing on testing, bug triage, release notes and other feature documentation.
As part the docs aspect of this, I've made a story in storyboard, and then added tasks to it as a I did a very fast (and incomplete) review of the existing text files that we tend to include under the umbrella of docs. There are 16 or so vague tasks which probably need work. https://storyboard.openstack.org/#!/story/2005190 I wanted to be sure to get these tasks written down, so I used storyboard, but I'm not sure that the way I did was the right choice, but it's start [1]. I hope that as group we can start addressing these next week. Feel free to assign a task to yourself if you're able, and add other tasks. On the member-of feature, I remain inclined to include it if we agree a) it is low impact, and b) that strict adherence to feature freeze in a project as simple as placement, though important, is less important than it is in something like nova. I agree with both of those statement, but we should figure out what the whole group thinks and go with that. I won't be at the meeting on Monday (funeral), but that ^ registers my thoughts on the matter. I hope people will figure something out and also make some doc and other release-related plans.
Also, I'm pretty sure there is nothing in either nova or blazar that is currently not going to merge in stein if this doesn't land in Stein, so based on that it can wait until Train IMO.
For me it has less to do with something waiting on it, and more to do with making things available as they happen.
As for the osc-placement change, I think we can also defer that to Train. It's been talked about since the Dublin PTG and is just now getting implemented, which is good, but also means it's apparently low priority for people and therefore can wait until Train (we won't have another osc-placement release in Stein anyway since clients are frozen now). Also, since it's a client, it will work with older versions of the placement API anyway, i.e. osc-placement in Train should work fine with placement from Queens, so again I don't see a rush here.
That's fine from my standpoint. My default position is if we have a chance to get more stuff out we should try, but if we don't, especially if there's not a lot of clamouring that's fine too. In fact clamouring-oriented-development is probably a good strategy.
There were no nibbles on last week's plea, so I'll say again: If anyone reading this is in a position to provide third party CI with fancy hardware for NUMA, NFV, FPGA, and GPU related integration testing with nova, there's a significant need for that.
There is some discussion happening in OpenLab about this:
Thanks for that, looks like some potential. I have no idea how people keeping track of all the change in our nearby universe lately. At least we have pupdates. [1] Adapting to storyboard is going to involve a lot of experimenting. I think we can make good use of it, but it is different from anything else I've used. To some extent it is an abstraction that one can manipulate as one likes. That makes it potentially powerful, but probably challenging for people who just want to report a bug. We will almost certainly need to do more triage than we've been used to in placement in the past. -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent tw: @anticdent
On 2019/03/09 2:35, Matt Riedemann wrote:
On 3/8/2019 8:51 AM, Chris Dent wrote:
We need to decide if we want to do what amounts to a feature freeze exception for Tetsuro's negative member-of handling linked below and work out how/when to fit Mel's allocation ratio work on osc-placement (also below) into the world so people can use it.
Probably not surprisingly I'm a -1 on pushing this stuff in after feature freeze. We have two weeks to RC1 and people should be focusing on testing, bug triage, release notes and other feature documentation. Also, I'm pretty sure there is nothing in either nova or blazar that is currently not going to merge in stein if this doesn't land in Stein, so based on that it can wait until Train IMO.
It's okay. I started this just because there was a possibility to make it happen before the feature freeze and I made this open anyway because there was no reason to hold it locally and it is good to look into the implementation in placement side before discussing the usage of this API such as in Nova [1]. [1] https://review.openstack.org/#/c/609960/
As for the osc-placement change, I think we can also defer that to Train. It's been talked about since the Dublin PTG and is just now getting implemented, which is good, but also means it's apparently low priority for people and therefore can wait until Train (we won't have another osc-placement release in Stein anyway since clients are frozen now). Also, since it's a client, it will work with older versions of the placement API anyway, i.e. osc-placement in Train should work fine with placement from Queens, so again I don't see a rush here.
There were no nibbles on last week's plea, so I'll say again: If anyone reading this is in a position to provide third party CI with fancy hardware for NUMA, NFV, FPGA, and GPU related integration testing with nova, there's a significant need for that.
There is some discussion happening in OpenLab about this:
Good to know. It would be nice. -- Tetsuro Nakamura <nakamura.tetsuro@lab.ntt.co.jp> NTT Network Service Systems Laboratories TEL:0422 59 6914(National)/+81 422 59 6914(International) 3-9-11, Midori-Cho Musashino-Shi, Tokyo 180-8585 Japan
participants (3)
-
Chris Dent
-
Matt Riedemann
-
TETSURO NAKAMURA