Open Stack

On Thu, Jul 23, 2015 at 2:40 AM, Derek Higgins <derekh at redhat.com> wrote:
> See below
>
>
> On 21/07/15 20:29, Derek Higgins wrote:
>>
>> Hi All,
>>     Something we discussed at the summit was to switch the focus of
>> tripleo's deployment method to deploy using instack using images built
>> with tripleo-puppet-elements. Up to now all the instack work has been
>> done downstream of tripleo as part of rdo. Having parts of our
>> deployment story outside of upstream gives us problems mainly because it
>> becomes very difficult to CI what we expect deployers to use while we're
>> developing the upstream parts.
>>
>> Essentially what I'm talking about here is pulling instack-undercloud
>> upstream along with a few of its dependency projects (instack,
>> tripleo-common, tuskar-ui-extras etc..) into tripleo and using them in
>> our CI in place of devtest.
>>
>> Getting our CI working with instack is close to working but has taken
>> longer then I expected because of various complications and distractions
>> but I hope to have something over the next few days that we can use to
>> replace devtest in CI, in a lot of ways this will start out by taking a
>> step backwards but we should finish up in a better place where we will
>> be developing (and running CI on) what we expect deployers to use.
>>
>> Once I have something that works I think it makes sense to drop the jobs
>> undercloud-precise-nonha and overcloud-precise-nonha, while switching
>> overcloud-f21-nonha to use instack, this has a few effects that need to
>> be called out
>>
>> 1. We will no longer be running CI on (and as a result not supporting)
>> most of the the bash based elements
>> 2. We will no longer be running CI on (and as a result not supporting)
>> ubuntu
>>
>> Should anybody come along in the future interested in either of these
>> things (and prepared to put the time in) we can pick them back up again.
>> In fact the move to puppet element based images should mean we can more
>> easily add in extra distros in the future.
>>
>> 3. While we find our feet we should remove all tripleo-ci jobs from non
>> tripleo projects, once we're confident with it we can explore adding our
>> jobs back into other projects again
>>
>> Nothing has changed yet, I order to check we're all on the same page
>> this is high level details of how I see things should proceed so shout
>> now if I got anything wrong or you disagree.
>
>
> Ok, I have a POC that has worked end to end in our CI environment[1], there
> are a *LOT* of workarounds in there so before we can merge it I need to
> clean up and remove some of those workarounds and todo that a few things
> need to move around, below is a list of what has to happen (as best I can
> tell)
>
> 1) Pull in tripleo-heat-template spec changes to master delorean
> We had two patches in the tripleo-heat-template midstream packaging that
> havn't made it into the master packaging, these are
> https://review.gerrithub.io/241056 Package firstboot and extraconfig
> templates
> https://review.gerrithub.io/241057 Package environments and newtork
> directories

I've merged these 2 (the ones against the correct branch, not the ones
you abandoned :-) )

>
> 2) Fixes for instack-undercloud (I didn't push these directly incase it
> effected people on old versions of puppet modules)
> https://github.com/rdo-management/instack-undercloud/pull/5

Can you submit this on gerrithub?:
https://review.gerrithub.io/#/q/project:rdo-management/instack-undercloud

>
> 3) Add packaging for various repositories into openstack-packaging
> I've pulled the packaging for 5 repositories into
> https://github.com/openstack-packages
> https://github.com/openstack-packages/python-ironic-inspector-client
> https://github.com/openstack-packages/python-rdomanager-oscplugin
> https://github.com/openstack-packages/tuskar-ui-extras
> https://github.com/openstack-packages/ironic-discoverd
> https://github.com/openstack-packages/tripleo-common
>
> I havn't imported these into gerrithub (incase following discussion we need
> to delete them again) but assuming we're in agreement we should pull them
> into gerrithub)
>
> 4) update rdoinfo
> https://github.com/redhat-openstack/rdoinfo/pull/69
> If everybody is happy with all above we should merge this, all of the
> packages needed will now be on the delorean master repository
>
> 5) Move DELOREAN_REPO_URL in tripleo-ci to a new delorean repo that includes
> all of the new packages
>
> 6) Take most of the workarounds out of this patch[1] and merge it
>
> 7) Reorg the tripleo ci tests (essentially remove all of the bash element
> based tests).

3 - 7 sound good to me.

>
> 8) Pull instack, instack-undercloud, python-rdomanager-oscplugin,
> triple-common, tuskar-ui-extras and maybe more into the upstream gerrit

+1, note that tripleo-common is already in gerrit/git.openstack.org
(http://git.openstack.org/cgit/openstack/tripleo-common)

>
> From here on the way to run the tripleo will be to follow the documentation
> in instack-undercloud, we should no longer be using devtest, this means
> we've lost the automation devtest gave us, so we will have to slowly build
> that up again. The main thing we have gained is that we will now be
> developing upstream all parts of how we expect deployers to use tripleo.

As far as the automation is concerned, RDO uses khaleesi:
https://github.com/redhat-openstack/khaleesi
khaleesi is a set of ansible playbooks around
instack-undercloud/python-rdomanager-oscplugin. There might be a bit
too much complexity there for what is effectively ~ lines 204 - 240 in
your tripleo-ci patch at [1], so I'm not saying we *need* to use it.
But, I also don't want to re-invent the wheel though where we don't
need to, so just something to keep in mind.

>
> - we will still have dependencies on tripleo-incubator we need to remove
> these (or move things into other repositories), essentially were finished
> with this process once we're no longer installing the "tripleo" package.

What about moving these things into tripleo-common when the time is
right? I think the main thing is devtest_testenv.sh, and the dependent
scripts it calls.

> - The new CI (as is) is running on Fedora jenkins nodes but building (and
> deploying) centos images, we also discussed that some people would want to
> develop on Fedora we will have to create a Fedora job as well (which I'm
> sure will involve the need for adding Fedora support into instack)
>
> Everything in the first 4 steps is ready to go now, once done we can
> investigate moving DELOREAN_REPO_URL and remove the workarounds from the CI
> patch.

Thanks for working on this.

>
> Thanks,
> Derek
>
> [1] - https://review.openstack.org/#/c/185151/
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
-- James Slagle
--