Open Stack

Latest news on what's going on with resource providers and the
placement API. I've made some adjustments in the structure of this
since last time[0]. The new structure tries to put the stuff we need to
talk about, including medium and long term planning, at the top and
move the stuff that is summaries of what's going on on gerrit towards
the bottom. I think we need to do this to enhance the opportunities for
asynchronous resolution of some of the topics on our plates. If we
keep waiting until the next meeting where we are all there at the same
time, stuff will sit for too long.

[0] http://lists.openstack.org/pipermail/openstack-dev/2016-November/107982.html

# Things to Think About

(Note that I'm frequently going to be wrong or at least incomplete
about the things I say here, because I'm writing off the top of my
head. Half the point of writing this is to get it correct by
collaborative action. If you see something that is wrong, please
shout out in a response. This section is for discussion of stuff that
isn't yet being tracked well or has vague conflicts.)

The general goal with placement for Ocata is to have both the nova
scheduler and resource tracker talking to the API to usefully limit
the number of hosts that the scheduler evaluates when selecting
destinations. There are several segments of work coming together to
make this possible, some of which are further along than others.

## Update Client Side to Consider Aggregates

When the scheduler requests a list of resource providers, that list
ought to include compute nodes that are are associated, via
aggregates, with any shared resource provides (such as shared disk)
that can satisfy the resource requirements in the request.

Meanwhile, when a compute node places a VM that uses shared disk the
allocation of resources made by the resource tracker need to go to
the right resource providers.

This is a thing we know we need to do but is not something for which
(as far as I know) we've articulated a clear plan or really started
on.

## Update Scheduler to Request Limited Resource Providers

The "Scheduler Filters in DB" spec[1] has merged along with its
pair, "Filter Resource Providers by Request"[2], and the work has
started[3].

There are some things to consider as that work progresses:

* The bit about aggregates in the previous section: the list of
   returned resource providers needs to include associated providers.
   To quote Mr. Pipes:

       we will only return resource providers to the scheduler that
       are compute nodes in Ocata. the resource providers that the
       placement service returns will either have the resources
       requested or will be associated with aggregates that have
       providers that match the requested resources.

* There is unresolved debate about the structure of the request being
   made to the API. Is it POST or a GET, does it have a body or use
   query strings? The plan is to resolve this discussion in the review
   of the code at [3].

[1] http://specs.openstack.org/openstack/nova-specs/specs/ocata/approved/resource-providers-scheduler-db-filters.html
[2] http://specs.openstack.org/openstack/nova-specs/specs/ocata/approved/resource-providers-get-by-request.html
[3] https://review.openstack.org/#/c/386242/

## Docs

In addition to needing an api-ref we also need a placement-dev.rst to
go alongside the placement.rst. The -dev would mostly explain the how
and the why of the placement API archicture, how the testing works,
etc. That's mostly on me.

## Placement Upgrade/Installation issues

(This is a straight copy from the previous message)

In his response[4] to this topic Matt R pointed out todos for this
topic:

* get the placement-api enabled by default in the various bits of
   ocata CI 
* ensure that microversions are being used on both sides of the
   placement API transactions (that's true in pending changes to
   both the API and the resource tracker)

[4] http://lists.openstack.org/pipermail/openstack-dev/2016-November/107177.html

## Long Term Stuff

### Making Claims in the Placement API

After Ocata the placement API will evolve to make claims, on the
/allocations endpoint. When presented with a set of resources
requirements _the_ resource provider that satisfies those requiements
will be returned and the claim of resources made in a single step. To
quote Mr. Pipes again:

     once we have a placement service actually doing claims, the
     returned resource providers for an allocation will be the actual
     resource providers that were allocated against (which include
     *both* compute node providers as well as any resource provider of
     a shared resource that was allocated)

Just so folk are aware.

### Moving Placement out of Nova

If this is something we ever plan to do (there appear to be multiple
points of view) then it is something we need to prepare for to ease
the eventual transition. Some of these things include:

* Removing as much 'nova.*' packages from the hierarchy of placement
   modules.
* Getting the new placement DB[5][6] handled in some way
* Removing remotable from the resource provider objects. The intent is
   that these will never be accessed other than through the HTTP API and
   since that scales horizontally, no RPC should be required. If we
   ever plan to remove it, sooner is better than later. A POC has been
   submitted[7], but there's disagreement about whether we should do
   it. We need to resolve that.

[5] https://review.openstack.org/#/c/362766/
[6] https://etherpad.openstack.org/p/placement-optional-db-spec
[7] https://review.openstack.org/#/c/404279/

# Pending Planned Work

## Custom Resource Classes

Jay just posted a big update[8] on that so go look at that. A lot of
code has merged, but a lot of code[9] is still in flight.

[8] http://lists.openstack.org/pipermail/openstack-dev/2016-December/108393.html
[9] https://review.openstack.org/#/q/topic:bp/custom-resource-classes

## Filtering compute nodes with the placement API

Already mentioned (with links) above.

## Nested Resource Providers

In discussions yesterday about Ocata priorities[10] we clarified that
while resource providers matter, they are a stretch for Ocata. The
primary goal is to have enough discussion and experimentation now so
that we can have useful discussions at the PTG.

The spec[11] has merged, the code is in a stack[12]. There's general
agreement on the implementation, but there still seems to be some
concern about how it is all going to work and what it all means in
actual practice. The expectation is that we'll figure things when
we're doing that actual practice.

[10] https://review.openstack.org/#/c/404456/
[11] http://specs.openstack.org/openstack/nova-specs/specs/ocata/approved/nested-resource-providers.html
[12] https://review.openstack.org/#/c/377138/

## Allocations for generic PCI devices

This code was abandoned because it was making some bad assumptions
about how PCI device handling is done. See the abandoned review[13]
for more information.

[13] https://review.openstack.org/#/c/374681/

# Pending Pickup Work

(Bugs[14], stuff from the leftovers etherpad[15], other random bits of
improvement.)

[14] https://bugs.launchpad.net/nova/+bugs?field.tag=placement
[15] https://etherpad.openstack.org/p/placement-newton-leftovers

* Demo inventory update script:
   https://review.openstack.org/#/c/382613/

   This one might be considered a WIP because how it chooses to do
   things (rather simply and dumbly) may not be in line with expecations.

* CORS support in placement API:
   https://review.openstack.org/#/c/392891/

   John Garbutt's review led to finding a huge bug in this (service
   wouldn't start in an actual deployment). That's been fixed.

* Handling limits in schema better
   https://review.openstack.org/#/c/399002/ (needs review)
   https://review.openstack.org/#/c/398998/ (needs fixes from submitter)

* [WIP] Placement api: Add json_error_formatter to defaults
   https://review.openstack.org/#/c/395194/

   This is an effort to avoid boilerplate, but no good solution has
   been determined yet. Reviewers can help us figure a good way to
   hande things.

* Small improvements to placement.rst
   https://review.openstack.org/#/c/403811/

# End

As usual, I hope this is useful to people. If something is missing
or incorrect please say so. It's quite a bit of work to assemble this,
but it's useful to me, so I'd be doing it anyway, even if I wasn't
sending it out. I hope other people find it useful. If there's
something I can do to make it more useful, let me know.

-- 
Chris Dent                 ¯\_(ツ)_/¯           https://anticdent.org/
freenode: cdent                                         tw: @anticdent