Open Stack

On Thu, 2014-01-16 at 15:37 +0000, Sullivan, Jon Paul wrote:
> > From: Jay Pipes [mailto:jaypipes at gmail.com]
> > On Thu, 2014-01-16 at 10:39 +0000, Sullivan, Jon Paul wrote:
> > > > From: Kyle Mestery [mailto:mestery at siliconloons.com]
> > > >	
> > > > FYI, here [1] are the meeting logs from today’s meeting.
> > > >
> > > > A couple of things have become apparent here:
> > > >
> > > > 1. No one has a working Neutron 3rd party testing rig yet which is
> > > > voting
> > > >     consistently. If I’ve missed something, please, someone correct
> > me.
> > > > 2. People are still hung on issues around Jenkins/gerrit
> > integration.
> > >
> > > This issue can be very easily resolved if people were to use Jenkins
> > Job Builder [2] for the creation of their Jenkins testing jobs.  This
> > would allow the reuse of simple macros already in existence to guarantee
> > correct configuration of Jenkins jobs at 3rd party sites.  This would
> > also allow simple reuse of the code used by the infra team to create the
> > openstack review and gate jobs, ensuring 3rd party testers can generate
> > the correct code from the gerrit change and also publish results back in
> > a standard way.
> > >
> > > I can't recommend Jenkins Job Builder highly enough if you use
> > Jenkins.
> > >
> > > [2] https://github.com/openstack-infra/jenkins-job-builder
> > 
> > ++ It's a life-saver. We used it heavily in AT&T with our
> > Gerrit/Jenkins/Zuul CI system.
> > 
> > -jay
> 
> It seems to me that shared JJB macros could be the most concise and simple way
> of describing 3rd party testing integration requirements.
> 
> So the follow-on questions are:
> 1. Can the 3rd party testing blueprint enforce, or at least link to,
>    use of specific JJB macros for integration to the openstack gerrit?
>       1a. Where should shared JJB code be stored?

Well, technically, this already exists. The openstack-infra/config
project already has pretty much everything a 3rd party would ever need
to setup an OpenStack environment, execute Tempest (or other) tests
against the environment, save and publish artifacts, and send
notifications of test results upstream.

> 2. Is it appropriate for 3rd party testers to share their tests as
>    JJB code, if they are willing?
>       2a. Would this live in the same location as (1a)?

Why would 3rd party testers be using anything other than Tempest for
integration testing? Put another way... if a 3rd party *is* using
something other than Tempest, why not put it in Tempest :)

> For those unfamiliar with JJB, here is a little example of what you might do:
> 
> Example of (untested) JJB macro describing how to configure Jenkins to
> trigger from gerrit:
> <snip>

As much as JJB is total awesomesauce -- as it prevents people needing to
manually update Jenkins job config.xml files -- any 3rd party that is
attempting to put together a test environment/platform for which you
intend to interact with the upstream CI system should go check out
devstack-gate [1], read the scripts, and grok it.

I'm working on some instructions to assist admins in 3rd party testing
labs in setting all of their platform up using the upstream tools like
devstack-gate and JJB, and this documentation should be done around
middle of next week. I'll post to the ML with links to that
documentation when it's done.

Best,
-jay

[1] https://github.com/openstack-infra/devstack-gate