Open Stack

On Tue, Aug 28, 2018 at 1:20 PM, Chris Dent <cdent+os at anticdent.org> 
wrote:
> On Mon, 27 Aug 2018, melanie witt wrote:
> 
>> I think we should use the openstack review system (gerrit) for 
>> moving the code. We're moving a critical piece of nova to its own 
>> repo and I think it's worth having the review and history contained 
>> in the openstack review system.
> 
> This seems a reasonable enough strategy, in broad strokes. I want to
> be sure that we're all actually in agreement on the details, as
> we've had a few false starts and I think some of the details are
> getting confused in the shuffle and the general busy-ness in progress.
> 
> Is anyone aware of anyone who hasn't commented yet that should? If
> you are, please poke them so we don't surprise them.
> 
>> Using smaller changes that make it easy to see import vs content 
>> changes might make review faster than fewer, larger changes.
> 
> I _think_ we ought to be able to use the existing commits from the
> runs-throughs-to-passing-tests already done, but if we use the
> strategy described below it doesn't really matter: the TDD approach
> (after fixing paths and test config) is pretty fast.
> 
>> The most important bit of all of this is making sure we don't break 
>> anything in the process for operators and users consuming nova and 
>> placement, and ensure the upgrade path from rocky => stein is 
>> tested in grenade.
> 
> This is one of the areas where pretty active support from all of
> nova will be required: getting zuul, upgrade paths, and the like
> clearly defined and executed.
> 
>> The steps I think we should take are:
>> 
>> 1. We copy the placement code into the openstack/placement repo and 
>> have it passing all of its own unit and functional tests.
> 
> To break that down to more detail, how does this look?
> (note the ALL CAPS where more than acknowledgement is requested)
> 
> 1.1 Run the git filter-branch on a copy of nova
>     1.1.1 Add missing files to the file list:
>           1.1.1.1 .gitignore
>           1.1.1.2 # ANYTHING ELSE?
> 1.2 Push -f that thing, acknowledge to be broken, to a seed repo on 
> github
>     (ed's repo should be fine)
> 1.3 Do the repo creation bits described in
>     https://docs.openstack.org/infra/manual/creators.html
>     to seed openstack/placement
>     1.3.1 set zuul jobs. Either to noop-jobs, or non voting basic
>     func and unit # INPUT DESIRED HERE

I suggest to add a non-voting unit and functional job and iterate on 
the repo to make them green, then turn them to voting.
I also think that we can add a non-voting tempest full job as well. 
Making it green depends on how hard to deploy placement from the new 
repo to tempest. I think as soon as placement repo has passing gabbits 
(e.g functional job) and we can deploy placement in tempest then 
tempest will be green soon.

> 1.4 Once the repo exists with some content, incrementally bring it to
>     working
>     1.4.1 Update tox.ini to be placement oriented
>     1.4.2 Update setup.cfg to be placement oriented
>     1.4.3 Correct .stesr.conf
>     1.4.4 Move base of placement to "right" place
>     1.4.5 Move unit and functionals to right place
>     1.4.6 Do automated path fixings
>     1.4.7 Set up translation domain and i18n.py corectly
>     1.4.8 Trim placement/conf to just the conf settings required
>           (api, base, database, keystone, paths, placement)
>     1.4.9 Remove database files that are not relevant (the db api is
>           not used by placement)
>     1.4.10 Fix the Database Fixture to be just one database
>     1.4.11 Disable migrations that can't work (because of
>            dependencies on nova code, 014 and 030 are examples)
>            # INPUT DESIRED HERE AND ON SCHEMA MIGRATIONS IN GENERAL
>     1.4.12 Incrementally get tests working
>     1.4.13 Fix pep8
> 1.5 Make zuul pep, unit and functional voting
> 1.6 Create tools for db table sync/create
> 1.7 Concurrently go to step 2, where the harder magic happens.
> 1.8 Find and remove dead code (there will be some).
> 1.9 Tune up and confirm docs
> 1.10 Grep for remaining "nova" (as string and spirit) and fix
> 
> 
> Item 1.4.12 may deserve some discussion. When I've done this the
> several times before, the strategy I've used is to be test driven:
> run either functional or unit tests, find and fix one of the errors
> revealed, commit, move on.
> 
> This strategy has worked very well for me because of the "test
> driven" part, but I'm hesitant to do it if reviewers are going to
> get to a patch and say "why didn't you also change X?" The answer to
> that question is "because this is incremental and test driven and
> the tests didn't demand that change (yet)". Sometimes that will mean
> that things of the same class of change are in different commits.
> 
> Are people okay with that and willing to commit to being okay with
> that answer in reviews? To some extent we need to have some faith on
> the end result: the tests work. If people are not okay with that, we
> need the people who are not to determine and prove the alternate
> strategy. I've had this one work and work well.

I like this test driven approach. If I start to leave comments like 
"why didn't you also change X?" in these patches then please point me 
to this mail and I will correct my behavior. :)
I think for me the trust towards the end result of these changes will 
come from the fact that the number of passing test cases are increases 
at every step.

> 
> Please help to refine the above, thank you.
> 
>> 2. We have a stack of changes to zuul jobs that show nova working 
>> but deploying placement in devstack from the new repo instead of 
>> nova's repo. This includes the grenade job, ensuring that upgrade 
>> works.
> 
> If we can make a list for this (and the subsequent) major items that
> is as detailed as I've made for step 1 above, I think that will help
> us avoid some of the confusion and frustration that comes up. I'm
> neither able nor willing to be responsible for creating those lists
> for all these points, but very happy to help.
> 

Let's collaborate on that list making. I added a list of jobs I foresee 
to the etherpad 
https://etherpad.openstack.org/p/placement-extract-stein-copy

Cheers,
gibi

>> 3. When those pass, we merge them, effectively orphaning nova's copy 
>> of placement. Switch those jobs to voting.
>> 
>> 4. Finally, we delete the orphaned code from nova (without needing 
>> to make any changes to non-placement-only test code -- code is 
>> truly orphaned).
> 
> In case you missed it, one of the things I did earlier in the
> discussion was make it so that the wsgi script for placement defined
> in nova's setup.cfg [1] could:
> 
> * continue to exist
> * with the same name
> * using the nova.conf file
> * running the extracted placement code
> 
> That was easy to do because of the work over the last year or so
> that has been hardening the boundary between placement and nova, in
> place. I've been assuming that maintaining the option to use
> original conf file is a helpful trick for people. Is that the case?
> 
> Thanks.
> 
> [1] 
> https://review.openstack.org/#/c/596291/3/nova/api/openstack/placement/wsgi.py
> --
> Chris Dent                       ٩◔̯◔۶           
> https://anticdent.org/
> freenode: cdent                                         tw: @anticdent
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: 
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev