Open Stack

Here's your placement and resource providers update 23. There wasn't
one last week because of summit. As usual, please let me know if
there is anything incorrect or missing.

# From Summit

Placement was all over the place at summit, coming up as a relevant
piece of the puzzle in many sessions.

Jay's sessions were well attended and the questions seemed to
indicate that people understood the potential. Videos:

* https://www.openstack.org/videos/boston-2017/scheduler-wars-a-new-hope
* https://www.openstack.org/videos/boston-2017/scheduler-wars-revenge-of-the-split

One (of three) of the VM and Baremetal sessions chose "cross project
placement" as a high priorty. See the "winners" section at the end
of:

     https://etherpad.openstack.org/p/BOS-forum-operating-vm-and-baremetal

# What Matters Most

Claims against the placement API remain the highest priorty. There's
plenty of other work in progress too which needs to advance. Lots of
links within.

# What's Changed

Besides advancing some of the code (notably in claims and the
api-ref), and the additional of some alternate approaches on doing
claims, many people are still digesting summit.

"get providers sharing capacity"
https://review.openstack.org/#/c/460798/ merged just before summit.
This is the first in a stack of changes to allow selecting resource
providers based on association in aggregates where one of the
members has a trait of MISC_SHARES_VIA_AGGREGATE. This doesn't do
anything yet in part because it is big stack and in part because
nothing is yet managing the addition of the trait and aggregate
creation.

# Help Wanted

(This section not changed since last time)

Areas where volunteers are needed.

* General attention to bugs tagged placement:
          https://bugs.launchpad.net/nova/+bugs?field.tag=placement

* Helping to create api documentation for placement (see the Docs
          section below).

* Helping to create and evaluate functional tests of the resource
   tracker and the ways in which it and nova-scheduler use the
   reporting client. For some info see

   https://etherpad.openstack.org/p/nova-placement-functional

   and talk to edleafe. He has a work in progress at:

   https://review.openstack.org/#/c/446123/

   that seeks input and assistance.

* Performance testing. If you have access to some nodes, some basic
   benchmarking and profiling would be very useful.

# Main Themes

## Claims in the Scheduler

Work has started on placement-claims blueprint:

     https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/placement-claims

There are two different tracks of that work. One explores doing the
claims from the conductor, one from within the scheduler. There's a
related email thread about the summit session on this topic which is
worth reviewing (along with the dicussions on gerrit):

     http://lists.openstack.org/pipermail/openstack-dev/2017-May/117028.html

There's an interesting mix of trying to maintain sane code and
architecture while optimizing for common cases, preserving existing
(but not necessarily ideal) behaviors, and protecting against
pathology. Fun times.

## Traits

The main API is in place. Debate raged on how best to manage updates
of standard os-traits. In the end a cache similar to the one used
for resource classes was created:

      https://review.openstack.org/#/c/462769/

The code at that review needs some tweaks before it can be merged,
but is almost there.

Work will be required at some point on filtering resource providers
based on traits, and adding traits to resource providers from the
resource tracker. There's been some discussion on that in the
reviews of shared providers (below) because it will be a part of
the same mass (MASS!) of SQL.

## Shared Resource Providers

This work (when finished) makes it possible (amongst other things)
for use of shared disk resources to be tracked correctly.

     https://review.openstack.org/#/q/status:open+topic:bp/shared-resources-pike

Reviewers should be aware that the patches, at least as of today,
are structured in a way that evolves from the current state to the
eventual desired state in a way that duplicates some effort and
code. This was done intentionally by Jay to make the testing and
review more incremental. It's probably best to read through the
entire stack before jumping to any conclusions.

## Docs

https://review.openstack.org/#/q/topic:cd/placement-api-ref

Several reviews are in progress for documenting the placement API.
This is likely going to take quite a few iterations as we work out
the patterns and tooling. But it's great to see the progress and
when looking at the draft rendered docs it makes placement feel like
a real thing™.

Find me (cdent) or Andrey (avolkov) if you want to help out or have
other questions.

## Nested Resource Providers

(This section has not changed since last week)

On hold while attention is given to traits and claims. There's a
stack of code waiting until all of that settles:

         https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/nested-resource-providers

## Ironic/Custom Resource Classes

(This section has not changed since last week)

There's a blueprint for "custom resource classes in flavors" that
describes the stuff that will actually make use of custom resource
classes:

        https://blueprints.launchpad.net/nova/+spec/custom-resource-classes-in-flavors

The spec has merged, but the implementation has not yet started.

Over in Ironic some functional and integration tests have started:

        https://review.openstack.org/#/c/443628/

There's also a spec in progress discussing ways to filter baremetal
nodes by tenant/project:

        https://review.openstack.org/#/c/415512/

# Other Code/Specs

* https://review.openstack.org/#/q/project:openstack/osc-placement
   Work has started on an osc-plugin that can provide a command
   line interface to the placement API.
   It's quite likely that this code is going to need to be adopted by
   someone new.

* https://review.openstack.org/#/c/457636/
   Devstack change to install that plugin

* https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:get-inventory
   Clean up the interface for getting inventory information from
   virt drivers.

* https://review.openstack.org/#/c/460147/
   Use DELETE inventories method in report client.  This is proving
   somewhat more complicated than initially expected. DELETE of
   inventories doesn't give us a new generation for the associated
   resource provider.

* https://review.openstack.org/#/c/460231/
   Use a specific error message for inventory in use, not just the db
   exception. Jay has identified that this too is somewhat more
   complicated than initially expected, because for the time being
   the message is being parsed client-side.

* https://review.openstack.org/#/c/458049/
   Add a test to ensure that placement microversions have no gaps
   when there is more than one handler for a URL. This ought to be a
   quick and easy merge.

* https://review.openstack.org/#/c/427200/
   Add a status check for legacy filters in nova-status.

* https://review.openstack.org/#/c/461494/
   Start the removal of the can_host column from the resource
   providers database table. We're no longer going to use it.  A
   trait will indicate shared providers.

* https://review.openstack.org/#/c/454426/
   Handle new hosts for updating instance info in scheduler
   Currently in merge conflict.

* https://review.openstack.org/#/c/453916/
   Don't send instance updates from compute if not using filter
   scheduler

I've left a few items off the list that have not received attention
for a few months. If they are truly important they will rise back up
again like zombies from the grave.

# End

Your reward is zombie protection spray.

-- 
Chris Dent                  ┬──┬◡ﾉ(° -°ﾉ)       https://anticdent.org/
freenode: cdent                                         tw: @anticdent