Open Stack

At the PTG we decided that while it was unlikely we could manage
extracting Placement to its own project during Rocky, it would be
useful to make incremental progress in that direction so the ground is
prepared for when we do get around to it. This means making sure there
are clear boundaries between what is "placement code" and what is
"nova code", limiting imports of "nova code" into "placement code",
and keeping (or moving) "placement code" under a single directory so
that an eventual lift and shift to a new repo can maintain history[1].

The placement etherpad for rocky has some additional info:
https://etherpad.openstack.org/p/nova-ptg-rocky-placement

I've already done a fair amount of experimentation around these ideas,
resulting in some blog posts [2] and some active reviews [3]. There's
a mix in those reviews of work to consolidate placement, and work to
make sure that while placement still exists in the nova hierarchy it
doesn't import nova code it doesn't want to use.

This leaves plenty of other things that need to happen:

* Migration strategies need to be determined, mostly for data, but
   also in general. It may be best to simply document the options and
   let people do what they like. One option is to simply carry on using
   the nova_api db, but this presents eventual problems for schema
   adjustments [4].

* There are functional tests which currently live at functional/db
   which tests the persistence layer handling the placement-related
   objects. These should probably move under
   functional/api/openstack/placement

* There are functional tests at api/openstack/placement that tests the
   scheduler report client (put there initially because they run the
   placement service using wsgi-intercept). These should be moved
   elsewhere.

* Resource class fields are used by both nova and placement (and
   eventually other things) so should probably get the same treatment
   as os-traits [5], so we need an os-resource-classes and adjustments
   in both placement and nova to use it. In the meantime, a pending
   patch [6] puts those fields at the top of nova. Switching to
   os-resource-classes will also allow us to remove the resource class
   cache, which is confusing to manage during this transition.

* We should experiment with strategies for how nova will do testing
   when placement is no longer in-repo. It should (dangerous word) be
   possible for placement to provide (or for nova to create) a fixture
   which is a wsgi-intercepted placement service with a real datastore
   (which is what is done now, but in-tree) but this is not something
   we traditionally do in functional tests, so it may be important to
   start migrating some functional tests (e.g., the stuff in
   test_servers) to integration (which could still be in nova's tree).

* Eventually the work of creating a new repo, establishing status as
   an official project, setting up translation handling, and creating a
   core team will need to happen, but that can be put off until a time
   when we are actually doing the extraction.

* All the things I'm forgetting. There's plenty.

As stated at the PTG these are not things I can complete by myself
(especially the things I'm forgetting).  Volunteers are welcome and
encouraged for the stuff above. Good first steps are reading the blog
posts linked below, and reviewing the patches linked below. This will
establish some of the issues and reveal things I'm forgetting.

Thanks to everyone who has provided feedback on this stuff, either at
the PTG, on the reviews and blogs posts, or elsewhere. Even though we
can't magically do the extraction _right now_ the process of
experimentation and refactoring is improving placement in place and
setting the foundation for doing it later.

The footnotes:

[1] Some incantations with 'git filter-branch' ought to allow this.

[2] Placement extraction related blog posts:
* A series on placement in a container (which helps identify
   boundaries):
   * https://anticdent.org/placement-container-playground.html
   * https://anticdent.org/placement-container-playground-2.html
   * https://anticdent.org/placement-container-playground-3.html
* Notes on extraction: https://anticdent.org/placement-extraction.html
* Notes on scale (which also helps to identify boundaires):
   https://anticdent.org/placement-scale-fun.html

[3] * Using a simplified, non-nova, FaultWrapper wsgi middleware:
       https://review.openstack.org/533752

     * Moving object, database and exception handing into the placement
       hierarchy:
       https://review.openstack.org/#/c/540049/

     * Refactoring wsgi-related modules to limit nova's presence in
       placement during the transition:
       https://review.openstack.org/#/c/533797/

     * Cleaning up db imports so that import database models doesn't
       import a big pile of nova:
       https://review.openstack.org/#/c/533797/

[4] We keep coming up with reasons to change the schema. The latest is
adding generations to consumers.

[5] https://pypi.python.org/pypi/os-traits

[6] https://review.openstack.org/#/c/540049/11/nova/rc_fields.py

-- 
Chris Dent                      (⊙＿⊙')         https://anticdent.org/
freenode: cdent                                         tw: @anticdent