Open Stack

Welcome to placement update 18-17. This is an expand update, meaning
I've gone searching for new stuff to add to the lists.

In other news: I'll be on holiday next week so there won't be one of
these next week, unless somebody else wants to do one.

# Most Important

A great deal of stuff is reliant on nested providers in allocation
candidates, so moving it forward is the most important. Next in line
are granular resource requests and consumer generations.

# What's Changed

A race condition in synchronizing os-traits has been corrected, by
doing the sync in an independent transaction. Code that handles the
"local-delete" situation and cleans up allocations has been merged.

# Bugs

* Placement related bugs not yet in progress:  https://goo.gl/TgiPXb
       17, +1 on last week
* In progress placement bugs: https://goo.gl/vzGGDQ
       8, -4 (woot!) on last week

# Specs

Total last week: 12. Now: 11 (because one was abandoned)

* https://review.openstack.org/#/c/549067/
          VMware: place instances on resource pool
          (using update_provider_tree)

* https://review.openstack.org/#/c/552924/
         Proposes NUMA topology with RPs

* https://review.openstack.org/#/c/544683/
         Account for host agg allocation ratio in placement

* https://review.openstack.org/#/c/552105/
         Support default allocation ratios

* https://review.openstack.org/#/c/438640/
         Spec on preemptible servers

* https://review.openstack.org/#/c/557065/
       Proposes Multiple GPU types

* https://review.openstack.org/#/c/555081/
       Standardize CPU resource tracking

* https://review.openstack.org/#/c/502306/
       Network bandwidth resource provider

* https://review.openstack.org/#/c/509042/
       Propose counting quota usage from placement

* https://review.openstack.org/#/c/560174/
     Add history behind nullable project_id and user_id

* https://review.openstack.org/#/c/559466/
     Return resources of entire trees in Placement

# Main Themes

## Nested providers in allocation candidates

Representing nested provides in the response to GET
/allocation_candidates is required to actually make use of all the
topology that update provider tree will report. That work is in
progress at:

        https://review.openstack.org/#/q/topic:bp/nested-resource-providers-allocation-candidates

## Mirror nova host aggregates to placement

This makes it so some kinds of aggregate filtering can be done
"placement side" by mirroring nova host aggregates into placement
aggregates.

      https://review.openstack.org/#/q/topic:bp/placement-mirror-host-aggregates

This is still in progress but took a little attention break while
nested provider discussions took up (and destroyed) brains.

## Consumer Generations

This allows multiple agents to "safely" update allocations for a
single consumer. The code is in progress:

       https://review.openstack.org/#/q/topic:bp/add-consumer-generation

This is moving along, but is encountering some debate over how best
to represent the data and flexibly deal with the at least 3
different ways we need to manage consumer information.

## Granular

Ways and means of addressing granular requests when dealing with
nested resource providers. Granular in this sense is grouping
resource classes and traits together in their own lumps as required.
Topic is:

      https://review.openstack.org/#/q/topic:bp/granular-resource-requests

# Extraction

I've created patches that adjust devstack and zuul config to use the
separate placement database connection.

    devstack: https://review.openstack.org/#/c/564180/
    zuul: https://review.openstack.org/#/c/564067/
    db connection: https://review.openstack.org/#/c/362766/

All of these things could merge without requiring any action by
anybody. Instead they allow people to use different connections, but
don't require it.

Jay has made a first pass at an os-resource-classes:

     https://github.com/jaypipes/os-resource-classes/

which I thought was potentially more heavyweight than required, but
other people should have a look too.

The other main issue in extraction is the placement unit and
functional tests have a lot of dependence on the fixtures and base
classes used in the nova unit and functional tests. For the time
being that is okay, but it would be useful to start unwinding that,
soon. Same will be true for config.

# Other

14 entries last week, 4 of those have merged but we've added some to
bring the total to: 17.

* https://review.openstack.org/#/c/546660/
          Purge comp_node and res_prvdr records during deletion of
          cells/hosts

* https://review.openstack.org/#/q/topic:bp/placement-osc-plugin-rocky
          A huge pile of improvements to osc-placement

* https://review.openstack.org/#/c/524425/
          General policy sample file for placement

* https://review.openstack.org/#/c/527791/
         Get resource provider by uuid or name (osc-placement)

* https://review.openstack.org/#/c/477478/
         placement: Make API history doc more consistent

* https://review.openstack.org/#/c/556669/
       Handle agg generation conflict in report client

* https://review.openstack.org/#/c/537614/
       Add unit test for non-placement resize

* https://review.openstack.org/#/c/493865/
       cover migration cases with functional tests

* https://review.openstack.org/#/q/topic:bug/1732731
     Bug fixes for sharing resource providers

* https://review.openstack.org/#/c/517757/
     WIP at granular in allocation candidates

* https://review.openstack.org/#/c/561315/
   support multiple member_of qparams

* https://review.openstack.org/#/q/topic:bug/1763907
   member_of with shared providers fixes

* https://review.openstack.org/#/q/topic:bp/placement-return-all-resources
   return resoruces of entire trees in placement

* https://review.openstack.org/#/q/topic:placement-test-base
   refactor base functional test for allocation candidates and
   resource providers

* https://review.openstack.org/#/q/topic:libvirt-report-local-disk-only-if-no-sharing
   sharing disk in libvirt

* https://review.openstack.org/#/c/535517/
   Move refresh time from report client to prov tree

* https://review.openstack.org/#/c/561770/
   PCPU resource class

# End

I've inevitably forgotten something in here. Please follow up with
anything you think should be added. Also note that there's loads of
stuff in here that is exactly the same as last week. One of the main
reasons for producing this report is to ensure that stuff gets
attention at least something of a linear fashion.

-- 
Chris Dent                       ٩◔̯◔۶           https://anticdent.org/
freenode: cdent                                         tw: @anticdent