Open Stack

----- Original Message -----
> From: "Monty Taylor" <mordred at inaugust.com>
> To: openstack-infra at lists.openstack.org
> Sent: Wednesday, April 24, 2013 11:19:17 AM
> Subject: Re: [OpenStack-Infra] Talking is the first step.
> 
> 
> 
> On 04/24/2013 11:07 AM, Brian Lamar wrote:
> > 
> > On Apr 24, 2013, at 10:50 AM, Dan Prince <dprince at redhat.com> wrote:
> > 
> >> 
> >> ----- Original Message -----
> >>> From: Brian Lamar <brian.lamar at rackspace.com>
> >> 
> >>> So the question again becomes how can we get XenServer into the
> >>> gate.
> >> 
> >> Hi Lamar,
> >> 
> >> I understand there is a long term goal to move towards larger scale
> >> testing on bare metal w/ TripleO, etc. I'm very interested in that
> >> myself too. In the short/mid term however perhaps using SmokeStack
> >> (which supports testing multiple/flexible configurations) would
> >> help cover the scenario you described below via pre-gate testing?
> > 
> > I knew I was missing something big when I had finished this email.
> > Absolutely. Maybe our first fight really is how we can get SmokeStack
> > (and thus XenServer testing) as part of the gate.
> > 
> > My understanding is that the main gripe is that it needs to be easy
> > for people to see what failed and for them to have access to the
> > nodes to troubleshoot failures. Is this what is holding things back?


> > I'd love to get as many specifics as possible so we can work on
> > getting things integrated. This is probably a question for
> > Jim/Monty.
> 
> There are a few issues around this:
> 
> a) puppet/packages in the gate is a HUGE issue that we do not have a
> solution for yet

Sure. This topic may warrant its own thread. But it does seem like any *more realistic* 3rd party deployments may have the same sorts of problems that we face with gating on puppet (which uses packages). Probably best to take this offline.

> b) reliability engineering - running the amount of tests that we run
> with the other system is a really big undertaking and there is a full
> time team of people working basically just on making it work.
> c) duplication of effort. This was the thing that made me yell at Dan
> originally - although I think we're friends now. But we don't want two
> things in the gate driving testing. We have zuul, and we have jenkins,
> and we add new features to them constantly to keep up with the needs of
> the gate. We don't want to add a second parallel system to a collection
> of tools that's already a huge undertaking to manage, because it would
> mean that everything we do to the zuul system would have to be
> duplicated in the non-zuul system.

The focus of SmokeStack has really become pre-gate testing testing w/ packages and config management. Find issues before they land (outside of the scope of the gate). Because it isn't a gate we can be a bit more agile about the configuration we run (while still being careful) and provide good feedback to the community. If your goal is to gate things I do agree with the gist of what Monty says above in that zuul is our gate.

Gating issues aside I do want to clarify a couple of things lest I be seen as someone who goes around duplicating CI things on the project by just reading this thread:

- SmokeStack was around before there even was a devstack gate or zuul. (The openstack Jenkins did exist... but didn't cut it for me initially)

- It is a big undertaking, I've been running a live system to test upstream since Cactus. :)

- Once upon a time I even maintained a job in the OpenStack Jenkins that basically did the same thing a SmokeStack job does (w/ packages and config mgmt). After feedback from the Diablo conference there wasn't really a "light at the end of the tunnel" as far as making this job gating. For that matter upstream packages seemed to be going out of style (we stopped maintaining the upstream PPA's etc).

So... SmokeStack still exists and I continue to find value in running 3rd party configuration/testing with it.

I'm not trying to dig up history or dwell on any of this. But I'm also not trying to duplicate things and I do think SmokeStack might be useful for this task in the short term being a bit more agile than our full gating solution.

> 
> Now don't get me wrong - I think smokestack has been a very useful tool
> for the community and I think it's great that Dan has provided that
> service for the community. But if we were to "integrate" it in to the
> gate, I'd be much more interested in porting the things that it runs on
> machines to the system that the OpenStack project already runs.
> 
> That still doesn't address the issues with adding packaging and config
> management to the gate though.
> 
> If smokestack is useful to you to get started on this task, then by all
> means use it! But in terms of gating, I'd really like for us to engage
> in the hard work of meeting all of the requirements that are out there
> in a way that's integrated with the work that's already being done.

Lamar:

I do think SmokeStack could be useful in the short term for this. It is probably less than a weeks worth of work to get these 4 machines upgraded to XenServer 6.1 and then you'd have your pre-gate test results for the most part. It is your call as to whether you think this is worth the effort.

Mid-term: If we tease out the packages and config management pieces of what I'm currently doing and replace it with whatever (Devstack or custom shell/venv deployment solution), and then move that job into Jenkins/Zuul I think you'll have your gate.

Long-term: I'd love to see a fully automated bare metal solution. TripleO, etc.

Dan