Open Stack

----- Original Message -----
> From: "Dan Prince" <dprince at redhat.com>
> To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org>
> Sent: Thursday, 6 August, 2015 1:12:42 PM
> Subject: Re: [openstack-dev] [TripleO] Moving instack upstream
> 
> On Thu, 2015-07-23 at 07:40 +0100, Derek Higgins wrote:
> > See below
> > 
> > On 21/07/15 20:29, Derek Higgins wrote:
> > > Hi All,
> > >     Something we discussed at the summit was to switch the focus of
> > > tripleo's deployment method to deploy using instack using images
> > > built
> > > with tripleo-puppet-elements. Up to now all the instack work has
> > > been
> > > done downstream of tripleo as part of rdo. Having parts of our
> > > deployment story outside of upstream gives us problems mainly
> > > because it
> > > becomes very difficult to CI what we expect deployers to use while
> > > we're
> > > developing the upstream parts.
> > > 
> > > Essentially what I'm talking about here is pulling instack
> > > -undercloud
> > > upstream along with a few of its dependency projects (instack,
> > > tripleo-common, tuskar-ui-extras etc..) into tripleo and using them
> > > in
> > > our CI in place of devtest.
> > > 
> > > Getting our CI working with instack is close to working but has
> > > taken
> > > longer then I expected because of various complications and
> > > distractions
> > > but I hope to have something over the next few days that we can use
> > > to
> > > replace devtest in CI, in a lot of ways this will start out by
> > > taking a
> > > step backwards but we should finish up in a better place where we
> > > will
> > > be developing (and running CI on) what we expect deployers to use.
> > > 
> > > Once I have something that works I think it makes sense to drop the
> > > jobs
> > > undercloud-precise-nonha and overcloud-precise-nonha, while
> > > switching
> > > overcloud-f21-nonha to use instack, this has a few effects that
> > > need to
> > > be called out
> > > 
> > > 1. We will no longer be running CI on (and as a result not
> > > supporting)
> > > most of the the bash based elements
> > > 2. We will no longer be running CI on (and as a result not
> > > supporting)
> > > ubuntu
> 
> One more side effect is that I think it also means we no longer have
> the capability to test arbitrary Zuul refspecs for projects like Heat,
> Neutron, Nova, or Ironic in our undercloud CI jobs. We've relied on the
> source-repositories element to do this for us in the undercloud and
> since most of the instack stuff uses packages I think we would loose
> this capability.
> 
> I'm all for testing with packages mind you... would just like to see us
> build packages for any projects that have Zuul refspecs inline, create
> a per job repo, and then use that to build out the resulting instack
> undercloud.
> 
> This to me is the biggest loss in our initial switch to instack
> undercloud for CI. Perhaps there is a middle ground here where instack
> (which used to support tripleo-image-elements itself) could still
> support use of the source-repositories element in one CI job until we
> get our package building processes up to speed?
> 
> /me really wants 'check experimental' to give us TripleO coverage for
> select undercloud projects
> 
> > > 
> > > Should anybody come along in the future interested in either of
> > > these
> > > things (and prepared to put the time in) we can pick them back up
> > > again.
> > > In fact the move to puppet element based images should mean we can
> > > more
> > > easily add in extra distros in the future.
> > > 
> > > 3. While we find our feet we should remove all tripleo-ci jobs from
> > > non
> > > tripleo projects, once we're confident with it we can explore
> > > adding our
> > > jobs back into other projects again
> > > 
> > > Nothing has changed yet, I order to check we're all on the same
> > > page
> > > this is high level details of how I see things should proceed so
> > > shout
> > > now if I got anything wrong or you disagree.
> > 
> > Ok, I have a POC that has worked end to end in our CI environment[1],
> > 
> > there are a *LOT* of workarounds in there so before we can merge it I
> > 
> > need to clean up and remove some of those workarounds and todo that a
> > 
> > few things need to move around, below is a list of what has to happen
> > 
> > (as best I can tell)
> > 
> > 1) Pull in tripleo-heat-template spec changes to master delorean
> > We had two patches in the tripleo-heat-template midstream packaging
> > that
> > havn't made it into the master packaging, these are
> > https://review.gerrithub.io/241056 Package firstboot and extraconfig
> > templates
> > https://review.gerrithub.io/241057 Package environments and newtork
> > directories
> > 
> > 2) Fixes for instack-undercloud (I didn't push these directly incase
> > it
> > effected people on old versions of puppet modules)
> > https://github.com/rdo-management/instack-undercloud/pull/5
> > 
> > 3) Add packaging for various repositories into openstack-packaging
> > I've pulled the packaging for 5 repositories into
> > https://github.com/openstack-packages
> > https://github.com/openstack-packages/python-ironic-inspector-client
> > https://github.com/openstack-packages/python-rdomanager-oscplugin
> > https://github.com/openstack-packages/tuskar-ui-extras
> > https://github.com/openstack-packages/ironic-discoverd
> > https://github.com/openstack-packages/tripleo-common
> > 
> > I havn't imported these into gerrithub (incase following discussion
> > we
> > need to delete them again) but assuming we're in agreement we should
> > pull them into gerrithub)
> > 
> > 4) update rdoinfo
> > https://github.com/redhat-openstack/rdoinfo/pull/69
> > If everybody is happy with all above we should merge this, all of the
> > 
> > packages needed will now be on the delorean master repository
> > 
> > 5) Move DELOREAN_REPO_URL in tripleo-ci to a new delorean repo that
> > includes all of the new packages
> 
> There is a puppet-neutron change I needed to use the latest Neutron
> packages BTW:
> https://review.openstack.org/#/c/207861/
> 
> Just a heads up if we rebase the Delorean URL to something more recent
> (last 2 weeks or so) that includes this neutron commit.
> 
> > 
> > 6) Take most of the workarounds out of this patch[1] and merge it
> > 
> > 7) Reorg the tripleo ci tests (essentially remove all of the bash
> > element based tests).
> > 
> > 8) Pull instack, instack-undercloud, python-rdomanager-oscplugin,
> > triple-common, tuskar-ui-extras and maybe more into the upstream
> > gerrit
> 
> I would really like to see us rename python-rdomanager-oscplugin. I
> think any project having the name "RDO" in it probably doesn't belong
> under TripleO proper. Looking at the project there are some distro
> specific things... but those are fairly encapsulated (or could be made
> so fairly easily).

I agree, it made sense as it was the entrypoint to RDO-Manager. However,
it could easily be called the python-tripleo-oscplugin or similar. The
changes would be really trivial, I can only think of one area that
may be distro specific.


> >  From here on the way to run the tripleo will be to follow the
> > documentation in instack-undercloud, we should no longer be using
> > devtest, this means we've lost the automation devtest gave us, so we
> > will have to slowly build that up again. The main thing we have
> > gained
> > is that we will now be developing upstream all parts of how we expect
> > 
> > deployers to use tripleo.
> > 
> > - we will still have dependencies on tripleo-incubator we need to
> > remove
> > these (or move things into other repositories), essentially were
> > finished with this process once we're no longer installing the
> > "tripleo"
> > package.
> > - The new CI (as is) is running on Fedora jenkins nodes but building
> > (and deploying) centos images, we also discussed that some people
> > would
> > want to develop on Fedora we will have to create a Fedora job as well
> > 
> > (which I'm sure will involve the need for adding Fedora support into
> > instack)
> 
> Yes. I would like to see instack support Fedora as well.
> 
> > 
> > Everything in the first 4 steps is ready to go now, once done we can
> > investigate moving DELOREAN_REPO_URL and remove the workarounds from
> > the
> > CI patch.
> > 
> > Thanks,
> > Derek
> > 
> > [1] - https://review.openstack.org/#/c/185151/
> > 
> > _____________________________________________________________________
> > _____
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubs
> > cribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>