Open Stack

HTML: https://anticdent.org/placement-update-18-19.html

This is placement update 18-19. 18 was skipped because I was on
holiday. With this issue I'm going to start cross-posting to my blog
to increase exposure, double up the archiving, and get the content
showing up on the OpenStack [Planet](http://planet.openstack.org/).

One upshot of this change is that the content will now be formatted
more fully as Markdown.

I'll be travelling next week so there won't be one of these for weeks
20 or 21, unless someone else feels like it.

# Most Important

We're continuing to hope that granular and nested resource providers
will be fully merged by Summit (a bit more than a week from now).
Not clear if this will happen as last I checked it seemed we have
multiple threads of changes in progress, many of which will merge
conflict with one another. But then again, I may be out of date,
it's been difficult to find all those threads while trying to catch
up this week.

If you're going to be at summit there are (at least) two
placement-related forum sessions:

* <https://etherpad.openstack.org/p/YVR-placement-extraction>
* <https://etherpad.openstack.org/p/YVR-cinder-placement>

Please add to those etherpads if you have thoughts.

Also a summit, Ed and Eric will be presenting [Placement, Present
and Future, in Nova and
Beyond](https://www.openstack.org/summit/vancouver-2018/summit-schedule/events/20813/placement-present-and-future-in-nova-and-beyond).

# What's Changed

Granular requests can now be made to GET /allocation_candidates
(meaning resourcesN and requiredN are now accepted). A bug with the
safe_connect handler masking real problems has been fixed. The spec
for [Network Bandwidth Resource
Provider](https://review.openstack.org/#/c/502306/) has finally
merged after a lot of thinking and discussion. The spec for [Return
resources of entire trees in
Placement](https://review.openstack.org/#/c/559466/) has merged.
This allows the inclusion of resource providers which are not
providing inventory, but are part of the current tree, in the provider
summaries of a /allocation_candidates response.

There are some new specs (see the end of the specs list, below)
which extend required traits handling to be able to say "I need
at least one of these traits".

# Bugs

* Placement related [bugs not yet in
   progress](https://goo.gl/TgiPXb): 16, -1 on two weeks ago
* [In progress placement bugs](https://goo.gl/vzGGDQ) 10, +2 two
   weeks ago

# Specs

Total two weeks ago: 11. Now: 13

* <https://review.openstack.org/#/c/549067/>
   VMware: place instances on resource pool (using update_provider_tree)

* <https://review.openstack.org/#/c/552924/>
   Proposes NUMA topology with RPs

* <https://review.openstack.org/#/c/544683/>
   Account for host agg allocation ratio in placement

* <https://review.openstack.org/#/c/552105/>
   Support default allocation ratios

* <https://review.openstack.org/#/c/438640/>
   Spec on preemptible servers

* <https://review.openstack.org/#/c/557065/>
   Proposes Multiple GPU types

* <https://review.openstack.org/#/c/555081/>
   Standardize CPU resource tracking

* <https://review.openstack.org/#/c/509042/>
   Propose counting quota usage from placement

* <https://review.openstack.org/#/c/560174/>
   Add history behind nullable project_id and user_id

* <https://review.openstack.org/#/c/565565/>
   update add-consumer-generation to focus on API

* <https://review.openstack.org/#/c/565730/>
   Placement: any traits in allocation_candidate query

* <https://review.openstack.org/#/c/565741/>
   Placement: support mixing required traits with any traits

* <https://review.openstack.org/#/c/559718/>
   [WIP] Support Placement in Cinder

# Main Themes

## Nested providers in allocation candidates

Unfortunately I'm really unclear on what the current state of this
is. If someone else can give a quick overview that would be
excellent. There's code in progress at both of the following topics,
some of it is old and in merge conflict:

* <https://review.openstack.org/#/q/topic:bp/nested-resource-providers-allocation-candidates>
* <https://review.openstack.org/#/q/topic:bp/nested-resource-providers>

## Mirror nova host aggregates to placement

This makes it so some kinds of aggregate filtering can be done
"placement side" by mirroring nova host aggregates into placement
aggregates.

* <https://review.openstack.org/#/q/topic:bp/placement-mirror-host-aggregates>

This is still in progress but is still on its attention break.

## Consumer Generations

This allows multiple agents to "safely" update allocations for a
single consumer. The code is in progress:

* <https://review.openstack.org/#/q/topic:bp/add-consumer-generation>

There's also a related change for ensuring [consumer
records](https://review.openstack.org/#/c/567678/).

## Granular

Ways and means of addressing granular requests when dealing with
nested resource providers. Granular in this sense is grouping
resource classes and traits together in their own lumps as required.
Topic is:

* <https://review.openstack.org/#/q/topic:bp/granular-resource-requests>

The sole active change in that now is work in progress to get it
working from the Nova side.

# Extraction

I've created patches that adjust devstack and zuul config to use the
separate placement database connection.

* [devstack](https://review.openstack.org/#/c/564180/)
* [zuul](https://review.openstack.org/#/c/564067/)
* [db connection](https://review.openstack.org/#/c/362766/)

All of these things could merge without requiring any action by
users. Instead they allow people to use different connections, but
don't require it.

Jay has made a first pass at an
[os-resource-classes](https://github.com/jaypipes/os-resource-classes/)
which I thought was potentially more heavyweight than required, but
other people should have a look too.

As mentioned above there will be a [forum
session](https://etherpad.openstack.org/p/YVR-placement-extraction)
about extraction.

In the meantime, some of the low hanging fruit on extraction is
duplicating and extracting to their own files the various fixtures
and base test classes that are required by both the functional and
unit tests. And making them not import from the nova hierarchy.

# Other

17 entries two weeks ago. 19 now.

Some of the older items in this list are not getting much attention.
That's a shame. The list is ordered the way it is on purpose.

* <https://review.openstack.org/#/c/546660/>
   Purge comp_node and res_prvdr records during deletion of
   cells/hosts

* <https://review.openstack.org/#/q/topic:bp/placement-osc-plugin-rocky>
   A huge pile of improvements to osc-placement

* <https://review.openstack.org/#/c/524425/>
   General policy sample file for placement

* <https://review.openstack.org/#/c/527791/>
   Get resource provider by uuid or name (osc-placement)

* <https://review.openstack.org/#/c/477478/>
   placement: Make API history doc more consistent

* <https://review.openstack.org/#/c/556669/>
   Handle agg generation conflict in report client

* <https://review.openstack.org/#/c/537614/>
   Add unit test for non-placement resize

* <https://review.openstack.org/#/c/493865/>
   cover migration cases with functional tests

* <https://review.openstack.org/#/q/topic:bug/1732731>
   Bug fixes for sharing resource providers

* <https://review.openstack.org/#/q/topic:bp/placement-return-all-resources>
   return resoruces of entire trees in placement

* <https://review.openstack.org/#/q/topic:libvirt-report-local-disk-only-if-no-sharing>
   sharing disk in libvirt

* <https://review.openstack.org/#/c/535517/>
   Move refresh time from report client to prov tree

* <https://review.openstack.org/#/c/561770/>
   PCPU resource class

* <https://review.openstack.org/#/c/565886/>
   Add nova-manage placement heal_allocations CLI

* <https://review.openstack.org/#/c/567506/>
   Add random sleep between retry calls to placement

* <https://review.openstack.org/#/c/566166/>
   rework how we pass candidate request information

* <https://review.openstack.org/#/c/564876/>
   add root parent NULL online migration

* <https://review.openstack.org/#/q/topic:bp/bandwidth-resource-provider>
   add resource_requests field to RequestSpec

* <https://review.openstack.org/#/q/topic:bp/alloc-candidates-member-of>
   member_of related cleanups

# End

Please followup with the many things I've missed.

-- 
Chris Dent                       ٩◔̯◔۶           https://anticdent.org/
freenode: cdent                                         tw: @anticdent