Open Stack

On Tue, Apr 23, 2013 at 10:52 AM, Brian Lamar <brian.lamar at rackspace.com> wrote:
> Long story short: I want to create environments, manage environments, and deploy code to environments under the auspices of the OpenStack Infrastructure team.
>
> Now, I guess the first question I'd like answered is: Is this goal under the purview of OSCI (do you have a nickname/shortname?)? Is it reasonable? Perhaps you can't answer that without elaboration. Here is some.
>
We typically go by "Infra" as our responsibilities have grown far
beyond just running Jenkins and Gerrit. I think it is reasonable. I
would like to see more testing happen upstream and in the open.
Working through the infra team seems to be the most direct way to make
this happen. Will it happen overnight? probably not, but we are very
receptive to getting new things going. eg the recent work to add
RedHat to our VM mix for python 2.6 testing.

> The devstack-gate is great -- but it is devstack nonetheless. We should have a devstack gate. We also need other, more realistic gates. You know this, I'm pretty sure we agree on this at least but to be honest I can't remember with everything that's been happening the past week.
>
> We seem to have already:
>
> devstack-gate
> a very nice, flexible gating system
>
>
> What I/we would like to help with:
>
> more gating scenarios
> deployment tool(s) standardization
>
>
> One issue is the vague term "more gating scenarios". Obviously I mean everything you've ever thought off. All permutations of $platform, $config_management, $packaging_method, $deployment_method, $use_cells I suppose. Since that's a lot, we'll choose the one we care about the most: XenServer, masterless puppet, venv, mcollective, 2 cells.
>
> Woah, woah, woah, you say. Where did masterless puppet, venv, and mcollective come from?! Well, those are what we're using now. If they have to change because they're not the technologies chosen for the future then so be it. I'm not attached to technologies.
>
At this point I think the most important thing would be to pick one
scenario and make it work. Once we have one figured out
adding/changing scenarios should be much simpler. Goal 1 being make
something work and Goal 2 aligning the tests with useful scenarios (I
think the OpenStack user committee has stats on what sort of
deployment scenarios are most popular. We could use that information
to guide us).

> However, if you ignore all of the above, a great first step is XenServer and forget all the other things (for now).
>
++ to starting small

> So the question again becomes how can we get XenServer into the gate. I imagine it going something like this:
>
> 1) Hardware is provided with XenServer 6.1 installed
> 2) Script for Jenkins will boot XenServer VM on hardware provided
> 3) VM will be configured using...devstack?
> 4) This script will be an unofficial gate (aka 3rd party gate) until it is considered stable
> 5) This script will be integrated into the official process either as a gate-gate or a periodic-gate
>
> As you can see I'm pretty fuzzy on the details, which is where this list comes in. Heck, some of my team is on this list and will probably be shouting at their monitors that I'm doing it all wrong. It isn't the first time and won't be the last time.
>
Let me start with what we have today. Currently we have access to two
cloud providers and exclusively use VMs that they provide for all
infrastructure and testing. If we can start with testing deployment
scenarios on top of these resources the barrier to entry will be much
lower. There is talk about getting dedicated hardware resources for
testing as well, but there are additional details to sort out before
that happens (on site support etc).

Our Jenkins jobs are entirely configurable through the files at
https://github.com/openstack-infra/config/tree/master/modules/openstack_project/files/jenkins_job_builder/config.
For dynamic slave creation we have the devstack-gate pool of nodes
(which have not actually had devstack installed on them before tests
are run so could be used for generic testing as well) and we sort of
kinda have jclouds jenkins plugin working for spinning up nodes. I
might be getting ahead of myself but if someone would be adventurous
and proposed a jenkins job (or jobs) to do a scenario test of some
sort that ran once a day (or on some interval) I think that would be
great. Doing so would give us something concrete to talk about.

As an alternative we could start doing this as a true 3rd party test
that reports results back to Gerrit. The nice thing about this
approach is that you can iterate without the need for upstream to
approve every little change. Then when you are satisfied with the
results we integrate it in one change. The downside to this approach
is I think it will make it harder for everyone involved to become
familiar with how upstream CI/Infra operates and learn the tools that
are available to them.

Now I am beginning to ramble :) It is great to see this getting more traction.

Clark