Open Stack

Sorry for the delay in following up on this but I've been on my hols

The instack ci jobs should now work, it appears to be getting hit by a 
LOT of network glitches in particular downloading from the centos 
infrastructure but I think to avoid more delay we should merge the 
things and work on improving reliability, We have 3 things to merge in 
order to start the move

1. Switch the f21 nonha job to use instack (eventually we'll remove the 
old codepath)
https://review.openstack.org/#/c/185151/

2. Remove most of our ci jobs (we can build back up again afterwards)
https://review.openstack.org/#/c/205479/

3. pull in 3 repositories upstream
https://review.openstack.org/#/c/215186/

There will be follow up patches after these but this will get the ball 
rolling

thanks,
Derek.

On 23/07/15 07:40, Derek Higgins wrote:
> See below
>
> On 21/07/15 20:29, Derek Higgins wrote:
>> Hi All,
>>     Something we discussed at the summit was to switch the focus of
>> tripleo's deployment method to deploy using instack using images built
>> with tripleo-puppet-elements. Up to now all the instack work has been
>> done downstream of tripleo as part of rdo. Having parts of our
>> deployment story outside of upstream gives us problems mainly because it
>> becomes very difficult to CI what we expect deployers to use while we're
>> developing the upstream parts.
>>
>> Essentially what I'm talking about here is pulling instack-undercloud
>> upstream along with a few of its dependency projects (instack,
>> tripleo-common, tuskar-ui-extras etc..) into tripleo and using them in
>> our CI in place of devtest.
>>
>> Getting our CI working with instack is close to working but has taken
>> longer then I expected because of various complications and distractions
>> but I hope to have something over the next few days that we can use to
>> replace devtest in CI, in a lot of ways this will start out by taking a
>> step backwards but we should finish up in a better place where we will
>> be developing (and running CI on) what we expect deployers to use.
>>
>> Once I have something that works I think it makes sense to drop the jobs
>> undercloud-precise-nonha and overcloud-precise-nonha, while switching
>> overcloud-f21-nonha to use instack, this has a few effects that need to
>> be called out
>>
>> 1. We will no longer be running CI on (and as a result not supporting)
>> most of the the bash based elements
>> 2. We will no longer be running CI on (and as a result not supporting)
>> ubuntu
>>
>> Should anybody come along in the future interested in either of these
>> things (and prepared to put the time in) we can pick them back up again.
>> In fact the move to puppet element based images should mean we can more
>> easily add in extra distros in the future.
>>
>> 3. While we find our feet we should remove all tripleo-ci jobs from non
>> tripleo projects, once we're confident with it we can explore adding our
>> jobs back into other projects again
>>
>> Nothing has changed yet, I order to check we're all on the same page
>> this is high level details of how I see things should proceed so shout
>> now if I got anything wrong or you disagree.
>
> Ok, I have a POC that has worked end to end in our CI environment[1],
> there are a *LOT* of workarounds in there so before we can merge it I
> need to clean up and remove some of those workarounds and todo that a
> few things need to move around, below is a list of what has to happen
> (as best I can tell)
>
> 1) Pull in tripleo-heat-template spec changes to master delorean
> We had two patches in the tripleo-heat-template midstream packaging that
> havn't made it into the master packaging, these are
> https://review.gerrithub.io/241056 Package firstboot and extraconfig
> templates
> https://review.gerrithub.io/241057 Package environments and newtork
> directories
>
> 2) Fixes for instack-undercloud (I didn't push these directly incase it
> effected people on old versions of puppet modules)
> https://github.com/rdo-management/instack-undercloud/pull/5
>
> 3) Add packaging for various repositories into openstack-packaging
> I've pulled the packaging for 5 repositories into
> https://github.com/openstack-packages
> https://github.com/openstack-packages/python-ironic-inspector-client
> https://github.com/openstack-packages/python-rdomanager-oscplugin
> https://github.com/openstack-packages/tuskar-ui-extras
> https://github.com/openstack-packages/ironic-discoverd
> https://github.com/openstack-packages/tripleo-common
>
> I havn't imported these into gerrithub (incase following discussion we
> need to delete them again) but assuming we're in agreement we should
> pull them into gerrithub)
>
> 4) update rdoinfo
> https://github.com/redhat-openstack/rdoinfo/pull/69
> If everybody is happy with all above we should merge this, all of the
> packages needed will now be on the delorean master repository
>
> 5) Move DELOREAN_REPO_URL in tripleo-ci to a new delorean repo that
> includes all of the new packages
>
> 6) Take most of the workarounds out of this patch[1] and merge it
>
> 7) Reorg the tripleo ci tests (essentially remove all of the bash
> element based tests).
>
> 8) Pull instack, instack-undercloud, python-rdomanager-oscplugin,
> triple-common, tuskar-ui-extras and maybe more into the upstream gerrit
>
>  From here on the way to run the tripleo will be to follow the
> documentation in instack-undercloud, we should no longer be using
> devtest, this means we've lost the automation devtest gave us, so we
> will have to slowly build that up again. The main thing we have gained
> is that we will now be developing upstream all parts of how we expect
> deployers to use tripleo.
>
> - we will still have dependencies on tripleo-incubator we need to remove
> these (or move things into other repositories), essentially were
> finished with this process once we're no longer installing the "tripleo"
> package.
> - The new CI (as is) is running on Fedora jenkins nodes but building
> (and deploying) centos images, we also discussed that some people would
> want to develop on Fedora we will have to create a Fedora job as well
> (which I'm sure will involve the need for adding Fedora support into
> instack)
>
> Everything in the first 4 steps is ready to go now, once done we can
> investigate moving DELOREAN_REPO_URL and remove the workarounds from the
> CI patch.
>
> Thanks,
> Derek
>
> [1] - https://review.openstack.org/#/c/185151/
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev