[placement] update 19-22
HTML: https://anticdent.org/placement-update-19-22.html Welcome to placement update 19-22. # Most Important We are continuing to work through issues associated with the [spec for nested magic](https://review.opendev.org/662191). Unsurprisingly, there are edge cases where we need to be sure we're doing the right thing, both in terms of satisfying the use cases as well as making sure we don't violate the general model of how things are supposed to work. # What's Changed * We've had a few responses on [the thread to determine the fate of can_split](http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006726.html). The consensus at this point is to not worry about workloads that mix NUMA-aware guests with non-NUMA-aware on the same host. * Support forbidden traits (microversion 1.22) has been added to osc-placement. * Office hours will be 1500 UTC on Wednesdays. * os-traits 0.13.0 and 0.14.0 were released. * Code to optionaly run a [wsgi profiler](https://docs.openstack.org/placement/latest/contributor/testing.html#profili...) in placement has merged. * The request group mapping in allocation candidates spec has merged, more on that in themes, below. # Specs/Features * <https://review.opendev.org/654799> Support Consumer Types. This has some open questions that need to be addressed, but we're still go on the general idea. * <https://review.opendev.org/662191> Spec for nested magic 1. The easier parts of nested magic: same_subtree, resource request groups, verbose suffixes (already merged as 1.33). Recently some new discussion here. These and other features being considered can be found on the [feature worklist](https://storyboard.openstack.org/#!/worklist/594). Some non-placement specs are listed in the Other section below. # Stories/Bugs (Numbers in () are the change since the last pupdate.) There are 20 (1) stories in [the placement group](https://storyboard.openstack.org/#!/project_group/placement). 0 (0) are [untagged](https://storyboard.openstack.org/#!/worklist/580). 3 (1) are [bugs](https://storyboard.openstack.org/#!/worklist/574). 4 (0) are [cleanups](https://storyboard.openstack.org/#!/worklist/575). 11 (0) are [rfes](https://storyboard.openstack.org/#!/worklist/594). 2 (0) are [docs](https://storyboard.openstack.org/#!/worklist/637). If you're interested in helping out with placement, those stories are good places to look. * Placement related nova [bugs not yet in progress](https://goo.gl/TgiPXb) on launchpad: 15 (-1). * Placement related nova [in progress bugs](https://goo.gl/vzGGDQ) on launchpad: 7 (0). # osc-placement osc-placement is currently behind by 11 microversions. Pending Changes: * <https://review.openstack.org/#/c/640898/> Add 'resource provider inventory update' command (that helps with aggregate allocation ratios). * <https://review.openstack.org/586056> Provide a useful message in the case of 500-error # Main Themes ## Nested Magic The overview of the features encapsulated by the term "nested magic" are in a [story](https://storyboard.openstack.org/#!/story/2005575). There is some in progress code, some of it WIPs to expose issues: * <https://review.opendev.org/663009> PoC: resourceless request, including some code from <https://review.opendev.org/657510> WIP: Allow RequestGroups without resources * <https://review.opendev.org/657463> Add NUMANetworkFixture for gabbits * <https://review.opendev.org/662785> Prepare objects for allocation request mappings. This work exposed a [bug in hash handling](https://storyboard.openstack.org/#!/story/2005822) that is [being fixed](https://review.opendev.org/663137). ## Consumer Types Adding a type to consumers will allow them to be grouped for various purposes, including quota accounting. A [spec](https://review.opendev.org/654799) has started. There are some questions about request and response details that need to be resolved, but the overall concept is sound. ## Cleanup We continue to do cleanup work to lay in reasonable foundations for the nested work above. As a nice bonus, we keep eking out additional performance gains too. * <https://review.opendev.org/661769> Add olso.middleware.cors to conf generator * <https://review.opendev.org/661922> Modernize CORS config and setup. Ed Leafe's ongoing work with using a graph database probably needs some kind of report or update. # Other Placement Miscellaneous changes can be found in [the usual place](https://review.opendev.org/#/q/project:openstack/placement+status:open). There are several [os-traits changes](https://review.opendev.org/#/q/project:openstack/os-traits+status:open) being discussed. # Other Service Users New discoveries are added to the end. Merged stuff is removed. Anything that has had no activity in 4 weeks has been removed. * <https://review.openstack.org/601596> Nova: spec: support virtual persistent memory * <https://review.openstack.org/#/q/topic:bug/1819923> Nova: nova-manage: heal port allocations * <https://review.openstack.org/650188> nova-spec: Allow compute nodes to use DISK_GB from shared storage RP * <https://review.opendev.org/659233> Cyborg: Placement report * <https://review.opendev.org/657884> Nova: Spec to pre-filter disabled computes with placement * <https://review.opendev.org/657801> rpm-packaging: placement service * <https://review.opendev.org/657016> Delete resource providers for all nodes when deleting compute service * <https://review.opendev.org/654066> nova fix for: Drop source node allocations if finish_resize fails * <https://review.opendev.org/656885> nova: WIP: Hey let's support routed networks y'all! * <https://review.opendev.org/662371> starlingx: Add placement chart patch to openstack-helm * <https://review.opendev.org/662229> helm: WIP: add placement chart * <https://review.opendev.org/662164> kolla-ansible: Add a explanatory note for "placement_api_port" * <https://review.opendev.org/658451> neutron-spec: L3 agent capacity and scheduling * <https://review.opendev.org/656023> Nova: Use OpenStack SDK for placement * <https://review.opendev.org/659302> puppet: Implement generic placement::config::placement_config * <https://review.opendev.org/663272> puppet: Add parameter for `scheduler/query_placement_for_image_type_support` * <https://review.opendev.org/612497> Nova: Spec: Provider config YAML file * <https://review.opendev.org/623558> Nova: single pass instance info fetch in host manager * <https://review.opendev.org/661245> Watcher: Add Placement helper * <https://review.opendev.org/663478> docs: Add Placement service to Minimal deployment for Stein * <https://review.opendev.org/663270> devstack: Add setting of placement microversion on tempest conf * <https://review.opendev.org/634551> libvirt: report pmem namespaces resources by provider tree * <https://review.opendev.org/657796> Nova: Defaults missing group_policy to 'none' * <https://review.opendev.org/660852> Nova: Remove PlacementAPIConnectFailure handling from AggregateAPI # End Making good headway. -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent
On Jun 7, 2019, at 7:17 AM, Chris Dent <cdent+os@anticdent.org> wrote:
Ed Leafe's ongoing work with using a graph database probably needs some kind of report or update.
Sure, be happy to. A few weeks ago I completed the code changes to remove sqlalchemy from the objects, and replace it with code to talk to the graph DB (Neo4j). One big issue is that there isn’t a 1:1 relationship between what is needed in the two database approaches. For example, there is no need to use the two-ID design that (IMO) overly complicates placement data. There is also no need to store the IDs of parents and root providers, but so much of the code depends on these values, I left them in there for now. One other twist is that an Allocation cannot exist without a Consumer, so all the code to handle the early microversions that support that was removed. I then moved on to getting the functional tests passing. Some early runs revealed holes in my understanding of what the code was supposed to be doing, so I fixed those. Most of the failures were in the tests/functional/db directory. I mentioned that to Chris in a side conversation, and he agreed that those tests would not be relevant, as the system had a completely different database, so I removed those. I tried to integrate Neo4j’s transaction model into the transaction framework of oslo_db and sqla, and while it works for the most part, it fails when running tox. I get “Invalid transaction” messages, which you would expect when one process closes another process’s transaction. Since the Python adapter I’m using (py2neo) creates a pool for connections, I suspect that the way tox runs is causing py2neo to reuse live connections. I haven’t had time to dig into this yet, but it is my current focus. When I run the tests individually, they pass without a problem. I am also planning on doing some performance measurement, so I guess I’ll look into the perfload stuff to see if it can work for this. One thing that is clear to me from all this work is that the way Placement is coded is very much a result of its relational DB roots. There were so many places I encountered when converting the code where I needed to do what felt like unnecessary steps to make the objects continue to work the way that are currently designed. Had this been a greenfield effort, the code for implementing Placement with a graph DB would have been much more direct and understandable. But the converted objects are working, and working well. -- Ed Leafe
participants (2)
-
Chris Dent
-
Ed Leafe