Open Stack

On Mon, 27 Aug 2018, melanie witt wrote:

> I think we should use the openstack review system (gerrit) for moving the 
> code. We're moving a critical piece of nova to its own repo and I think it's 
> worth having the review and history contained in the openstack review system.

This seems a reasonable enough strategy, in broad strokes. I want to
be sure that we're all actually in agreement on the details, as
we've had a few false starts and I think some of the details are
getting confused in the shuffle and the general busy-ness in progress.

Is anyone aware of anyone who hasn't commented yet that should? If
you are, please poke them so we don't surprise them.

> Using smaller changes that make it easy to see import vs content changes 
> might make review faster than fewer, larger changes.

I _think_ we ought to be able to use the existing commits from the
runs-throughs-to-passing-tests already done, but if we use the
strategy described below it doesn't really matter: the TDD approach
(after fixing paths and test config) is pretty fast.

> The most important bit of all of this is making sure we don't break anything 
> in the process for operators and users consuming nova and placement, and 
> ensure the upgrade path from rocky => stein is tested in grenade.

This is one of the areas where pretty active support from all of
nova will be required: getting zuul, upgrade paths, and the like
clearly defined and executed.

> The steps I think we should take are:
>
> 1. We copy the placement code into the openstack/placement repo and have it 
> passing all of its own unit and functional tests.

To break that down to more detail, how does this look?
(note the ALL CAPS where more than acknowledgement is requested)

1.1 Run the git filter-branch on a copy of nova
     1.1.1 Add missing files to the file list:
           1.1.1.1 .gitignore
           1.1.1.2 # ANYTHING ELSE?
1.2 Push -f that thing, acknowledge to be broken, to a seed repo on github
     (ed's repo should be fine)
1.3 Do the repo creation bits described in
     https://docs.openstack.org/infra/manual/creators.html
     to seed openstack/placement
     1.3.1 set zuul jobs. Either to noop-jobs, or non voting basic
     func and unit # INPUT DESIRED HERE
1.4 Once the repo exists with some content, incrementally bring it to
     working
     1.4.1 Update tox.ini to be placement oriented
     1.4.2 Update setup.cfg to be placement oriented
     1.4.3 Correct .stesr.conf
     1.4.4 Move base of placement to "right" place
     1.4.5 Move unit and functionals to right place
     1.4.6 Do automated path fixings
     1.4.7 Set up translation domain and i18n.py corectly
     1.4.8 Trim placement/conf to just the conf settings required
           (api, base, database, keystone, paths, placement)
     1.4.9 Remove database files that are not relevant (the db api is
           not used by placement)
     1.4.10 Fix the Database Fixture to be just one database
     1.4.11 Disable migrations that can't work (because of
            dependencies on nova code, 014 and 030 are examples)
            # INPUT DESIRED HERE AND ON SCHEMA MIGRATIONS IN GENERAL
     1.4.12 Incrementally get tests working
     1.4.13 Fix pep8
1.5 Make zuul pep, unit and functional voting
1.6 Create tools for db table sync/create
1.7 Concurrently go to step 2, where the harder magic happens.
1.8 Find and remove dead code (there will be some).
1.9 Tune up and confirm docs
1.10 Grep for remaining "nova" (as string and spirit) and fix


Item 1.4.12 may deserve some discussion. When I've done this the
several times before, the strategy I've used is to be test driven:
run either functional or unit tests, find and fix one of the errors
revealed, commit, move on.

This strategy has worked very well for me because of the "test
driven" part, but I'm hesitant to do it if reviewers are going to
get to a patch and say "why didn't you also change X?" The answer to
that question is "because this is incremental and test driven and
the tests didn't demand that change (yet)". Sometimes that will mean
that things of the same class of change are in different commits.

Are people okay with that and willing to commit to being okay with
that answer in reviews? To some extent we need to have some faith on
the end result: the tests work. If people are not okay with that, we
need the people who are not to determine and prove the alternate
strategy. I've had this one work and work well.

Please help to refine the above, thank you.

> 2. We have a stack of changes to zuul jobs that show nova working but 
> deploying placement in devstack from the new repo instead of nova's repo. 
> This includes the grenade job, ensuring that upgrade works.

If we can make a list for this (and the subsequent) major items that
is as detailed as I've made for step 1 above, I think that will help
us avoid some of the confusion and frustration that comes up. I'm
neither able nor willing to be responsible for creating those lists
for all these points, but very happy to help.

> 3. When those pass, we merge them, effectively orphaning nova's copy of 
> placement. Switch those jobs to voting.
>
> 4. Finally, we delete the orphaned code from nova (without needing to make 
> any changes to non-placement-only test code -- code is truly orphaned).

In case you missed it, one of the things I did earlier in the
discussion was make it so that the wsgi script for placement defined
in nova's setup.cfg [1] could:

* continue to exist
* with the same name
* using the nova.conf file
* running the extracted placement code

That was easy to do because of the work over the last year or so
that has been hardening the boundary between placement and nova, in
place. I've been assuming that maintaining the option to use
original conf file is a helpful trick for people. Is that the case?

Thanks.

[1] https://review.openstack.org/#/c/596291/3/nova/api/openstack/placement/wsgi.py
-- 
Chris Dent                       ٩◔̯◔۶           https://anticdent.org/
freenode: cdent                                         tw: @anticdent