Open Stack

20th placement and resource providers update! As usual: If I've
forgotten something, please let me know, either in a response here
or find me in IRC (as cdent).

If you're going to be in Boston there are some placement related
sessions that may be worth your while:

* Scheduler Wars: A New Hope
   https://www.openstack.org/summit/boston-2017/summit-schedule/events/17501/scheduler-wars-a-new-hope

* Behind the Scenes with Placement and Resource Tracking in Nova
   https://www.openstack.org/summit/boston-2017/summit-schedule/events/17511/behind-the-scenes-with-placement-and-resource-tracking-in-nova

# What Matters Most

Still some remaining issues in the claims via the scheduler spec. We
keep encountering snags here and there which need to be addressed.
See below for links and more detail.

# What's Changed

We've made progress on one issue which was stalling the claims
scheduler, only to find some more. The docs are moving along well.
Plenty of code down in "Other Code/Specs" is active and in need of
review. The placement API can now run using uwsgi and systemd in
devstack.

# Help Wanted

Areas where volunteers are needed.

* General attention to bugs tagged placement:
       https://bugs.launchpad.net/nova/+bugs?field.tag=placement

* Helping to create api documentation for placement (see the Docs
       section below).

* Helping to create and evaluate functional tests of the resource
       tracker and the ways in which it and nova-scheduler use the
       reporting client. For some info see
       https://etherpad.openstack.org/p/nova-placement-functional
       and talk to edleafe. He has a work in progress at:

       https://review.openstack.org/#/c/446123/

       that seeks input and assistance.

* Performance testing. If you have access to some nodes, some basic
      benchmarking and profiling would be very useful. See the
      performance section below.

# Main Themes

## Traits

The main API is in place, there's one patch left for a new command
to sync the os-traits library into the database:

     https://review.openstack.org/#/c/450125/

Work will be required at some point on filtering resource providers
based on traits, and adding traits to resource providers from the
resource tracker.

There is a stack of changes to the os-traits library to add more traits
and also automate creating symbols associated with the trait
strings:

     https://review.openstack.org/#/c/448283/

## Ironic/Custom Resource Classes

There's a blueprint for "custom resource classes in flavors" that
describes the stuff that will actually make use of custom resource
classes:

     https://blueprints.launchpad.net/nova/+spec/custom-resource-classes-in-flavors

The spec has merged, but the implementation has not yet started.

Over in Ironic some functional and integration tests have started:

     https://review.openstack.org/#/c/443628/

There's also a spec in progress discussing ways to filter baremetal
nodes by tenant/project:

     https://review.openstack.org/#/c/415512/

## Claims in the Scheduler

Progress has been made on the spec for claims in the scheduler:

       https://review.openstack.org/#/c/437424/

Continued eyes and brains required as we find edge cases that impact
how we're going to handle claims. One issue is how to deal with the
concept of instance overhead and where in the system this
information should be counted. There are a variety solutions, each
on a different place on the "works well enough" to "strictly
correct" spectrum.

Thinking about claims in the scheduler has also revealed some places
where it's possible for allocations to become wrong or orphaned:

       https://bugs.launchpad.net/nova/+bug/1679750
       https://bugs.launchpad.net/nova/+bug/1661312

## Shared Resource Providers

https://blueprints.launchpad.net/nova/+spec/shared-resources-pike

Progress on this will continue once traits and claims have moved forward.

## Nested Resource Providers

On hold while attention is given to traits and claims. There's a
stack of code waiting until all of that settles:

      https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/nested-resource-providers

## Docs

https://review.openstack.org/#/q/topic:cd/placement-api-ref

Several reviews are in progress for documenting the placement API.
This is likely going to take quite a few iterations as we work out
the patterns and tooling. But it's great to see the progress and
when looking at the draft rendered docs it makes placement feel like
a real thing™.

We need multiple reviewers on this stuff, early in the process, as
it helps to identify missteps in the phrasing and styling before we
develop bad habits.

Find me (cdent) or Andrey (avolkov) if you want to help out or have
other questions.

## Performance

We're aware that there are some redundancies in the resource tracker
that we'd like to clean up

         http://lists.openstack.org/pipermail/openstack-dev/2017-January/110953.html

but it's also the case that we've done no performance testing on the
placement service itself.

We ought to do some testing to make sure there aren't unexpected
performance drains.

# Other Code/Specs

* https://review.openstack.org/#/c/448791/
      Idempotent PUT for resource classes. This is something that was
      discovered while evaluating some resource tracker code. Stacked
      with it is code to allow the resource tracker to use it.

* https://review.openstack.org/#/q/project:openstack/osc-placement
      Work has started on an osc-plugin that can provide a command
      line interface to the placement API.

* https://review.openstack.org/#/c/457782/
     Implementation of get_inventory() for libvirt. This puts the
     guts of how to create the inventory of a resource provider in
     the virt driver (a good thing). We'll want similar things for
     all the virt drivers, eventually.

* https://review.openstack.org/#/c/455710/
     Fix the inspection of CONTENT_LENGTH in the placement handler.
     It was making a bad assumption that nginx+uwsgi revealed.

* https://review.openstack.org/#/c/458049/
     Add a test to ensure that placement microversions have no gaps
     when there is more than one handler for a URL.

* https://bugs.launchpad.net/nova/+bug/1632852
      Cache headers not produced by placement API. This was assigned to
      several different people over time, but I'm not sure if there is
      any active code.

* https://etherpad.openstack.org/p/placement-newton-leftovers
      There's still some lingering stuff on here, some of which is
      mentioned elsewhere in this message, but not all.

# End

I feel like there are probably things I've forgotten, but I hope the
above list is sufficient to keep people busy. Thanks for reading.

-- 
Chris Dent                 ¯\_(ツ)_/¯           https://anticdent.org/
freenode: cdent                                         tw: @anticdent