Open Stack

On 11/13/2013 05:14 AM, Thierry Carrez wrote:
> I have a few remarks (mostly around our enforcement of grenade
> testing, since there is little we can do once the project is
> "integrated"), but I don't want to start a larger discussion on the
> -tc list... Could you repost this to -dev first ?
> 
> (rationale: we must be careful not to have "closed" discussions on the
> -tc list so that the TC is not seen as a kabbale / ivory tower -- so
> the -tc list should be limited to TC meeting organization
> administrativia and posting pointers to other threads of interest to
> TC members)

Sure, absolutely. Reposting on -dev now.

> 
> Thanks,
> 
> Sean Dague wrote:
>> There were a few chats at summit about this, mostly on the infra / 
>> devstack / qa side of the house. Consider the following a straw man
>> to explain the current state of the world, and what I think a
>> starting point for discussion would be. I call out projects by name
>> here, not to make fun of them, but that I think concrete examples
>> bring the point home much more than abstracts (at least for me).
> 
>> This is looking at raising the bar quite a bit along the way.
>> However, as someone that spends a lot of time trying to keep the
>> whole ball of wax holding together, and is spending a lot of time
>> retroactively trying to get projects into our integrated gate (and
>> huge pain to everyone, as their gate commits get bounced by racey
>> projects), I think we really need to up this bar if we want a 20
>> integrated project version of OpenStack to hold together.
> 
>> ===================================================== Proposed new
>> Incubation and Graduation Requirements 
>> =====================================================
> 
> 
>> The Current State of the World ============================== 
>> OpenStack is currently 9 integrated projects in Havana, 10 once we
>> get to Icehouse (+Trove).
> 
>> Heat added tests to our integrated gate during the Havana cycle, 
>> though the deeper guest testing is not part of our gate.
>> Ceilometer has been integrated for 2 releases, and doesn't have
>> integrated gate testing. Trove is just now exploring how they'd get
>> into the gate for Icehouse release.
> 
>> Upgrade testing situation is worse. Only 5 projects currently are
>> part of upgrade testing: Nova, Cinder, Swift, Glance, Keystone. The
>> rest are not set up in that model at all, so we have no
>> infrastructure to ensure the other projects don't break upgrades of
>> themselves.
> 
>> Heat, Trove, and Savana represent an interesting challenge from 
>> current gate models in that real validation requires something
>> beyond a trivial guest, as they care about the contents inside
>> compute resources. This desire is likely to grow with other "Layer
>> 4" projects coming into the OpenStack ecosystem [1].
> 
>> Ironic as an incubated project provides an even different set of 
>> challenges, as not only do we not have a gating approach, but we
>> also really should have an approach to test an upgrade from
>> nova-baremetal => ironic to ensure there is a migration path for
>> people in the future.
> 
>> Ceilometer once relied on a version of MongoDB that worked in the 
>> gate, but they were never gating, so now they rely on a version of 
>> MongoDB that doesn't work in the gate. They made a technical
>> decision to upgrade requirements with no gate feedback because they
>> weren't actually integration testing on a regular basis.
> 
> 
>> Proposed Incubation requirements ================================ 
>> Once something becomes an integrated project, it's important that
>> they are able to run in the gate.
> 
>> Both devstack and devstack-gate now support hooks, so with a couple
>> of days of work any project in stackforge could build a gate job
>> which sets up a devstack of their configuration, including their
>> code, running some project specific test they feel is appropriate
>> to ensure they could run in the gate environment.
> 
>> This would ensure an incubated project works with OpenStack global 
>> requirements, or if it requires something new, that's known very 
>> clearly before incubation.
> 
> 
>> Proposed Graduation requirements ================================ 
>> All integrated projects should be in the integrated gate, as this
>> is the only way we provably know that they can all work together,
>> at the same level of requirements, in a consistent way.
> 
>> During incubation landing appropriate tests in Tempest is fair 
>> game. So the expectation would be that once a project is incubated 
>> they would be able to land tests in tempest. Before integrated
>> we'd need to ensure the project had tests which could take part in
>> the integrated gate, so as soon as a project is voted integrated,
>> it has some working integrated gate tests. (Note: there is actually
>> a symmetric complexity here, to be worked out later).
> 
> 
>> Proposed Stable Release requirements 
>> ==================================== We have this automatic
>> transition that happens when a project that's integrated for a
>> release, actually releases as part of that. I.e. Trove and
>> Icehouse. There is no additional TC decision about whether or not
>> Trove is part of the stable release, once integrated, it just is.
>> Nothing that it does over that cycle will kick it out of the stable
>> release. This is one of the reasons it needs to be in the
>> integrated gate **before** graduation.
> 
>> Additionally, upgrade path is critically important to our users,
>> and the number one piece of feedback we received from the User
>> Survey. It was also important enough to our developers that it was
>> scattered all over the Icehouse Design Summit. All integrated
>> projects should be included in upgrade testing the moment they are
>> in a stable release. (ex: when Icehouse is released, Trove should
>> be in master grenade, and upgrade testing from Icehouse -> master
>> for the J cycle from day one).
> 
> 
> 
>> TL;DR ===== - To get to incubated you must have a devstack-gate
>> job - To get to integrated you must have useful tests in the
>> integrated gate - To get into the stable/release you must have
>> upgrade testing enabled
> 
>> Raised Questions ================ - what about existing incubated
>> projects, what would be their time frame to get with this new
>> program - what about existing integrated projects that currently
>> don't exist with either an upgrade or gate story? - what about an
>> upgrade deprecation path (i.e. nova-network => neutron,
>> nova-baremetal => ironic)
> 
> 
>> 1. 
>> http://hackstack.org/x/blog/2013/09/05/openstack-seven-layer-dip-as-a-service/
> 
> 
> (the layers pattern is useful from a testing requirements model)
> 
> 
> 
> 
>> _______________________________________________ OpenStack-TC
>> mailing list OpenStack-TC at lists.openstack.org 
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-tc
> 
> 
> 
> 
> _______________________________________________
> OpenStack-TC mailing list
> OpenStack-TC at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-tc
> 

-- 
Sean Dague
http://dague.net

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-tc/attachments/20131113/f4cb0fba/attachment-0001.pgp>