Open Stack

Frankly I'm surprised that we have projects which aren't gating now. I
find that very concerning. We certainly shouldn't add any more.

Another issue -- how do we get the currently non-gating projects gating?

Michael

On Wed, Nov 13, 2013 at 10:41 AM, Sean Dague <sean at dague.net> wrote:
> There were a few chats at summit about this, mostly on the infra /
> devstack / qa side of the house. Consider the following a straw man to
> explain the current state of the world, and what I think a starting
> point for discussion would be. I call out projects by name here, not to
> make fun of them, but that I think concrete examples bring the point
> home much more than abstracts (at least for me).
>
> This is looking at raising the bar quite a bit along the way. However,
> as someone that spends a lot of time trying to keep the whole ball of
> wax holding together, and is spending a lot of time retroactively trying
> to get projects into our integrated gate (and huge pain to everyone, as
> their gate commits get bounced by racey projects), I think we really
> need to up this bar if we want a 20 integrated project version of
> OpenStack to hold together.
>
> =====================================================
>  Proposed new Incubation and Graduation Requirements
> =====================================================
>
>
> The Current State of the World
> ==============================
> OpenStack is currently 9 integrated projects in Havana, 10 once we get
> to Icehouse (+Trove).
>
> Heat added tests to our integrated gate during the Havana cycle,
> though the deeper guest testing is not part of our gate. Ceilometer
> has been integrated for 2 releases, and doesn't have integrated gate
> testing. Trove is just now exploring how they'd get into the gate for
> Icehouse release.
>
> Upgrade testing situation is worse. Only 5 projects currently are part
> of upgrade testing: Nova, Cinder, Swift, Glance, Keystone. The rest
> are not set up in that model at all, so we have no infrastructure to
> ensure the other projects don't break upgrades of themselves.
>
> Heat, Trove, and Savana represent an interesting challenge from
> current gate models in that real validation requires something beyond
> a trivial guest, as they care about the contents inside compute
> resources. This desire is likely to grow with other "Layer 4" projects
> coming into the OpenStack ecosystem [1].
>
> Ironic as an incubated project provides an even different set of
> challenges, as not only do we not have a gating approach, but we also
> really should have an approach to test an upgrade from nova-baremetal
> => ironic to ensure there is a migration path for people in the
> future.
>
> Ceilometer once relied on a version of MongoDB that worked in the
> gate, but they were never gating, so now they rely on a version of
> MongoDB that doesn't work in the gate. They made a technical decision
> to upgrade requirements with no gate feedback because they weren't
> actually integration testing on a regular basis.
>
>
> Proposed Incubation requirements
> ================================
> Once something becomes an integrated project, it's important that they
> are able to run in the gate.
>
> Both devstack and devstack-gate now support hooks, so with a couple of
> days of work any project in stackforge could build a gate job which
> sets up a devstack of their configuration, including their code,
> running some project specific test they feel is appropriate to ensure
> they could run in the gate environment.
>
> This would ensure an incubated project works with OpenStack global
> requirements, or if it requires something new, that's known very
> clearly before incubation.
>
>
> Proposed Graduation requirements
> ================================
> All integrated projects should be in the integrated gate, as this is
> the only way we provably know that they can all work together, at the
> same level of requirements, in a consistent way.
>
> During incubation landing appropriate tests in Tempest is fair
> game. So the expectation would be that once a project is incubated
> they would be able to land tests in tempest. Before integrated we'd
> need to ensure the project had tests which could take part in the
> integrated gate, so as soon as a project is voted integrated, it has
> some working integrated gate tests. (Note: there is actually a
> symmetric complexity here, to be worked out later).
>
>
> Proposed Stable Release requirements
> ====================================
> We have this automatic transition that happens when a project that's
> integrated for a release, actually releases as part of
> that. I.e. Trove and Icehouse. There is no additional TC decision
> about whether or not Trove is part of the stable release, once
> integrated, it just is. Nothing that it does over that cycle will kick
> it out of the stable release. This is one of the reasons it needs to
> be in the integrated gate **before** graduation.
>
> Additionally, upgrade path is critically important to our users, and
> the number one piece of feedback we received from the User Survey. It
> was also important enough to our developers that it was scattered all
> over the Icehouse Design Summit. All integrated projects should be
> included in upgrade testing the moment they are in a stable
> release. (ex: when Icehouse is released, Trove should be in master
> grenade, and upgrade testing from Icehouse -> master for the J cycle
> from day one).
>
>
>
> TL;DR
> =====
> - To get to incubated you must have a devstack-gate job
> - To get to integrated you must have useful tests in the integrated
>   gate
> - To get into the stable/release you must have upgrade testing enabled
>
> Raised Questions
> ================
> - what about existing incubated projects, what would be their time
>   frame to get with this new program
> - what about existing integrated projects that currently don't exist
>   with either an upgrade or gate story?
> - what about an upgrade deprecation path (i.e. nova-network =>
>   neutron, nova-baremetal => ironic)
>
>
> 1.
> http://hackstack.org/x/blog/2013/09/05/openstack-seven-layer-dip-as-a-service/
>    (the layers pattern is useful from a testing requirements model)
>
>
> --
> Sean Dague
> http://dague.net
>
>
> _______________________________________________
> OpenStack-TC mailing list
> OpenStack-TC at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-tc
>



-- 
Rackspace Australia