Open Stack

Placement and resource providers update 21. Please let me know if
anything is incorrect or missing.

If you're going to be in Boston there are some placement related
sessions that may be worth your while:

* Scheduler Wars: A New Hope
     https://www.openstack.org/summit/boston-2017/summit-schedule/events/17501/scheduler-wars-a-new-hope

* Scheduler Wars: Revenge of the Split
     https://www.openstack.org/summit/boston-2017/summit-schedule/events/17511/scheduler-wars-revenge-of-the-split

* Behind the Scenes with Placement and Resource Tracking in Nova
     https://www.openstack.org/summit/boston-2017/summit-schedule/events/17511/behind-the-scenes-with-placement-and-resource-tracking-in-nova

* Comparing Kubernetes and OpenStack Resource Management
     https://www.openstack.org/summit/boston-2017/summit-schedule/events/18726/comparing-kubernetes-and-openstack-resource-management

(I guess we'll have to see "NUMA Strikes Back" some other time.)

Next week there will be no scheduler subteam meeting, nor a
placement and resource providers update but efforts will be made to
summarize placement-related stuff that happens at the Forum.

# What Matters Most

Progress has begun on dealing with claims against the placement API.
Engaging with that is the top priority. There's plenty of other work
in progress too which needs to advance. Lots of links within.

# What's Changed

In addition to the work on claims, work has started on managing
resources that are shared via aggregates. When fully operational
this will finally allow correct consumption of shared disk!

Idempotent PUT for resource classes merged, which raises the max
microversion: https://review.openstack.org/#/c/448791/ .

# Help Wanted

(This section not changed since last week)

Areas where volunteers are needed.

* General attention to bugs tagged placement:
         https://bugs.launchpad.net/nova/+bugs?field.tag=placement

* Helping to create api documentation for placement (see the Docs
         section below).

* Helping to create and evaluate functional tests of the resource
         tracker and the ways in which it and nova-scheduler use the
         reporting client. For some info see
         https://etherpad.openstack.org/p/nova-placement-functional
         and talk to edleafe. He has a work in progress at:

         https://review.openstack.org/#/c/446123/

         that seeks input and assistance.

* Performance testing. If you have access to some nodes, some basic
        benchmarking and profiling would be very useful. See the
        performance section below.

# Main Themes

## Claims in the Scheduler

Work has started on placement-claims blueprint:

      https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/placement-claims

We intentionally left some detail out of the spec because we knew
that we would find some edge cases while the implementation is
explored.

## Traits

The main API is in place. Debate raged on how best to manage updates
of standard os-traits. In the end a cache similar to the one used
for resource classes was created:

     https://review.openstack.org/#/c/462769/

Work will be required at some point on filtering resource providers
based on traits, and adding traits to resource providers from the
resource tracker. There's been some discussion on that in the
reviews of shared providers (below) because it will be a part of
the same mass (MASS!) of SQL.

## Shared Resource Providers

https://blueprints.launchpad.net/nova/+spec/shared-resources-pike

Work and review on this is in progress at:

      https://review.openstack.org/#/q/status:open+topic:bp/shared-resources-pike

Reviewers should be aware that the patches, at least as of today,
are structured in a way that evolves from the current state to the
eventual desired state in a way that duplicates some effort and
code. This was done intentionally by Jay to make the testing and
review more incremental. It's probably best to read through the
entire stack before jumping to any conclusions. I know that I got
very concerned and confused by some of the duplication until I was
informed that it's just part of the process: the end goal ought to
be pretty clean.

## Docs

https://review.openstack.org/#/q/topic:cd/placement-api-ref

Several reviews are in progress for documenting the placement API.
This is likely going to take quite a few iterations as we work out
the patterns and tooling. But it's great to see the progress and
when looking at the draft rendered docs it makes placement feel like
a real thing™.

We need multiple reviewers on this stuff, early in the process, as
it helps to identify missteps in the phrasing and styling before we
develop bad habits. We've also found some ways in which the general
style of the docs can be improved to say more about when particular
errors might happen. We'll likely need more constructive use of
inclusions.

Find me (cdent) or Andrey (avolkov) if you want to help out or have
other questions.

## Performance

We're aware that there are some redundancies in the resource tracker
that we'd like to clean up

     http://lists.openstack.org/pipermail/openstack-dev/2017-January/110953.html

but it's also the case that we've done no performance testing on the
placement service itself. The request profile of the resource
tracker is going to change as a result of claims work, but not be
completely remedied.

We ought to do some testing to make sure there aren't unexpected
performance drains.

## Nested Resource Providers

(This section has not changed since last week)

On hold while attention is given to traits and claims. There's a
stack of code waiting until all of that settles:

        https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/nested-resource-providers

## Ironic/Custom Resource Classes

(This section has not changed since last week)

There's a blueprint for "custom resource classes in flavors" that
describes the stuff that will actually make use of custom resource
classes:

       https://blueprints.launchpad.net/nova/+spec/custom-resource-classes-in-flavors

The spec has merged, but the implementation has not yet started.

Over in Ironic some functional and integration tests have started:

       https://review.openstack.org/#/c/443628/

There's also a spec in progress discussing ways to filter baremetal
nodes by tenant/project:

       https://review.openstack.org/#/c/415512/

# Other Code/Specs

* https://review.openstack.org/#/q/project:openstack/osc-placement
        Work has started on an osc-plugin that can provide a command
        line interface to the placement API.

* https://review.openstack.org/#/c/457636/
        Devstack change to install that plugin

* https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:get-inventory
        Clean up the interface for getting inventory information from
        virt drivers.

* https://review.openstack.org/#/c/460147/
        Use DELETE inventories method in report client.
        This is proving somewhat more complicated than initially
        expected. DELETE of inventories doesn't give us a new
        generation for the associated resource provider.

* https://review.openstack.org/#/c/460231/
        Use a specific error message for inventory in use, not just
        the db exception. Jay has identified that this too is
        somewhat more complicated than initially expected, because
        for the time being the message is being parsed client-side.

* https://review.openstack.org/#/c/458049/
       Add a test to ensure that placement microversions have no gaps
       when there is more than one handler for a URL.

* https://review.openstack.org/#/c/427200/
       Add a status check for legacy filters in nova-status.

* https://review.openstack.org/#/c/461494/
       Start the removal of the can_host column from the resource
       providers database table. We're no longer going to use it.
       A trait will indicate shared providers.

* https://review.openstack.org/#/c/454426/
      Handle new hosts for updating instance info in scheduler

* https://review.openstack.org/#/c/453916/
      Don't send instance updates from compute if not using filter
      scheduler

* https://bugs.launchpad.net/nova/+bug/1632852
        Cache headers not produced by placement API. This was assigned to
        several different people over time, but I'm not sure if there is
        any active code.

* https://etherpad.openstack.org/p/placement-newton-leftovers
        There's still some lingering stuff on here, some of which is
        mentioned elsewhere in this message, but not all.

I suspect there's more, if I missed something, please tell me.

# End

Instead of a cookie, this time you get beer.

-- 
Chris Dent                  ┬──┬◡ﾉ(° -°ﾉ)       https://anticdent.org/
freenode: cdent                                         tw: @anticdent