Open Stack

HTML: https://anticdent.org/placement-update-18-40.html

Here's this week's placement update. We remain focused on
specs and pressing issues with extraction, mostly because until the
extraction is "done" in some form doing much other work is a bit
premature.

# Most Important

There have been several discussions recently about what to do with
options that impact both scheduling and configuration. Some of this
was in the thread about [intended purposes of
traits](http://lists.openstack.org/pipermail/openstack-dev/2018-October/thread.html#135301),
but more recently there was discussion on how to support guests
that want an HPET. Chris Friesen [summarized a
hangout](http://lists.openstack.org/pipermail/openstack-dev/2018-October/135446.html)
that happened yesterday that will presumably be reflected in an
[in-progress spec](https://review.openstack.org/#/c/607989/1).

The work to get [grenade upgrading to
placement](https://review.openstack.org/#/c/604454/) is very close.
After several iterations of tweaking, the grenade jobs are now
passing. There are still some adjustments to get devstack jobs
working, but the way is relatively clear. More on this in
"extraction" below, but the reason this is a most important is that
this stuff allows us to do proper integration and upgrade testing,
without which it is hard to have confidence.

# What's Changed

In both placement and nova, placement is no longer using
`get_legacy_facade()`. This will remove some annoying deprecation
warnings.

The nova->placement database migration script for MySQL has merged.
The postgresql version is still [up for
review](https://review.openstack.org/#/c/604028/).

Consumer generations are now being used in some allocation handling
in nova.

# Questions

* What should we do about nova calling the placement db, like in
   [nova-manage](https://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L416)
   and
   [nova-status](https://github.com/openstack/nova/blob/master/nova/cmd/status.py#L254).

* Should we consider starting a new extraction etherpad? The [old
   one](https://etherpad.openstack.org/p/placement-extract-stein-3)
   has become a bit noisy and out of date.

# Bugs

* Placement related [bugs not yet in progress](https://goo.gl/TgiPXb): 17.
   -1.
* [In progress placement bugs](https://goo.gl/vzGGDQ) 8. -1.

# Specs

Many of these specs don't seem to be getting much attention. Can the
dead ones be abandoned?

* <https://review.openstack.org/#/c/544683/>
   Account for host agg allocation ratio in placement
   (Still in rocky/)

* <https://review.openstack.org/#/c/595236/>
   Add subtree filter for GET /resource_providers

* <https://review.openstack.org/#/c/597601/>
   Resource provider - request group mapping in allocation candidate

* <https://review.openstack.org/#/c/549067/>
   VMware: place instances on resource pool
   (still in rocky/)

* <https://review.openstack.org/#/c/555081/>
   Standardize CPU resource tracking

* <https://review.openstack.org/#/c/599957/>
   Allow overcommit of dedicated CPU
   (Has an alternative which changes allocations to a float)

* <https://review.openstack.org/#/c/600016/>
   List resource providers having inventory

* <https://review.openstack.org/#/c/593475/>
   Bi-directional enforcement of traits

* <https://review.openstack.org/#/c/599598/>
   allow transferring ownership of instance

* <https://review.openstack.org/#/c/591037/>
   Modelling passthrough devices for report to placement

* <https://review.openstack.org/#/c/509042/>
   Propose counting quota usage from placement and API database
   (A bit out of date but may be worth resurrecting)

* <https://review.openstack.org/#/c/603585/>
   Spec: allocation candidates in tree

* <https://review.openstack.org/#/c/603805/>
   [WIP] generic device discovery policy

* <https://review.openstack.org/#/c/603955/>
   Nova Cyborg interaction specification.

* <https://review.openstack.org/#/c/601596/>
   supporting virtual NVDIMM devices

* <https://review.openstack.org/#/c/603352/>
   Spec: Support filtering by forbidden aggregate

* <https://review.openstack.org/#/c/552924/>
   Proposes NUMA topology with RPs

* <https://review.openstack.org/#/c/552105/>
   Support initial allocation ratios

* <https://review.openstack.org/#/c/569011/>
   Count quota based on resource class

* <https://review.openstack.org/#/c/607989/>
   WIP: High Precision Event Timer (HPET) on x86 guests

* <https://review.openstack.org/#/c/571111/>
   Add support for emulated virtual TPM

* <https://review.openstack.org/#/c/510235/>
   Limit instance create max_count (spec) (has some concurrency
   issues related placement)

* <https://review.openstack.org/#/c/141219/>
   Adds spec for instance live resize

So many specs.

# Main Themes

## Making Nested Useful

Work on getting nova's use of nested resource providers happy and
fixing bugs discovered in placement in the process. This is creeping
ahead. There is plenty of discussion going along nearby with regards
to various ways they are being used, notably GPUs.

* <https://review.openstack.org/#/q/topic:bp/use-nested-allocation-candidates>

I feel like I'm missing some things in this area. Please let me know
if there are others. This is related:

* <https://review.openstack.org/#/c/589085/>
   Pass allocations to virt drivers when resizing

## Extraction

There continue to be three main tasks in regard to placement
extraction:

1. upgrade and integration testing
2. database schema migration and management
3. documentation publishing

The upgrade aspect of (1) is in progress with a [patch to
grenade](https://review.openstack.org/#/c/604454/) and a [patch to
devstack](https://review.openstack.org/#/c/600162/). This is very
close to working. The remaining failures are with jobs that do not
have `openstack/placement` in `$PROJECTS`.

Once devstack is happy then we can start thinking about integration
testing using tempest. I've started some experiments with [using
gabbi](https://review.openstack.org/#/c/601614/) for that. I've
explained my reasoning in [a blog
post](https://anticdent.org/gabbi-in-the-gate.html).

Successful devstack is dependent on us having a reasonable solution
to (2). For the moment [a hacked up
script](https://review.openstack.org/#/c/600161/) is being used to
create tables. This works, but is not sufficient for deployers nor
for any migrations we might need to do.

Moving to alembic seems a reasonable thing to do, as a part of that.

We have work in progress to tune up the documentation but we are not
yet publishing documentation (3). We need to work out a plan for
this. Presumably we don't want to be publishing docs until we are
publishing code, but the interdependencies need to be teased out.

# Other

Going to start highlighting some specific changes across several
projects. If you're aware of something I'm missing, please let me
know.

* <https://review.openstack.org/#/c/601866/>
   Generate sample policy in placement directory
   (This is a bit stuck on not being sure what the right thing to do
   is.)

* <https://review.openstack.org/#/q/topic:reduce-complexity+status:open>
   Some efforts by Eric to reduce code complexity

* <https://review.openstack.org/#/q/topic:bp/initial-allocation-ratios>
   Improve handling of default allocation ratios

* <https://review.openstack.org/#/q/topic:minimum-bandwidth-allocation-placement-api>
   Neutron minimum bandwidth implementation

* <https://review.openstack.org/#/c/607953/>
   TripleO: Use valid_interfaces instead of os_interface for placement

* <https://review.openstack.org/#/c/605507/>
   Puppet: Separate placement database is not deprecated

* <https://review.openstack.org/#/c/602160/>
   Add OWNERSHIP $SERVICE traits

* <https://review.openstack.org/#/c/604182/>
   Puppet: Initial cookiecutter and import from nova::placement

* <https://review.openstack.org/#/c/601407/>
   WIP: Add placement to devstack-gate PROJECTS

* <https://review.openstack.org/#/c/586960/>
   zun: Use placement for unified resource management

# End

I'm going to be away next week, so if any my pending code needs some
fixes and is blocking other stuff, please fix it. Also, there will
be no pupdate next week (unless someone else does one).

-- 
Chris Dent                       ٩◔̯◔۶           https://anticdent.org/
freenode: cdent                                         tw: @anticdent