Open Stack

On 11/12/2013 07:17 PM, Robert Collins wrote:
> +1

+1

> On 13 November 2013 12:41, Sean Dague <sean at dague.net> wrote:
>> There were a few chats at summit about this, mostly on the infra /
>> devstack / qa side of the house. Consider the following a straw man to
>> explain the current state of the world, and what I think a starting
>> point for discussion would be. I call out projects by name here, not to
>> make fun of them, but that I think concrete examples bring the point
>> home much more than abstracts (at least for me).
>>
>> This is looking at raising the bar quite a bit along the way. However,
>> as someone that spends a lot of time trying to keep the whole ball of
>> wax holding together, and is spending a lot of time retroactively trying
>> to get projects into our integrated gate (and huge pain to everyone, as
>> their gate commits get bounced by racey projects), I think we really
>> need to up this bar if we want a 20 integrated project version of
>> OpenStack to hold together.
>>
>> =====================================================
>>  Proposed new Incubation and Graduation Requirements
>> =====================================================
>>
>>
>> The Current State of the World
>> ==============================
>> OpenStack is currently 9 integrated projects in Havana, 10 once we get
>> to Icehouse (+Trove).
>>
>> Heat added tests to our integrated gate during the Havana cycle,
>> though the deeper guest testing is not part of our gate. Ceilometer
>> has been integrated for 2 releases, and doesn't have integrated gate
>> testing. Trove is just now exploring how they'd get into the gate for
>> Icehouse release.
>>
>> Upgrade testing situation is worse. Only 5 projects currently are part
>> of upgrade testing: Nova, Cinder, Swift, Glance, Keystone. The rest
>> are not set up in that model at all, so we have no infrastructure to
>> ensure the other projects don't break upgrades of themselves.
>>
>> Heat, Trove, and Savana represent an interesting challenge from
>> current gate models in that real validation requires something beyond
>> a trivial guest, as they care about the contents inside compute
>> resources. This desire is likely to grow with other "Layer 4" projects
>> coming into the OpenStack ecosystem [1].
>>
>> Ironic as an incubated project provides an even different set of
>> challenges, as not only do we not have a gating approach, but we also
>> really should have an approach to test an upgrade from nova-baremetal
>> => ironic to ensure there is a migration path for people in the
>> future.
>>
>> Ceilometer once relied on a version of MongoDB that worked in the
>> gate, but they were never gating, so now they rely on a version of
>> MongoDB that doesn't work in the gate. They made a technical decision
>> to upgrade requirements with no gate feedback because they weren't
>> actually integration testing on a regular basis.
>>
>>
>> Proposed Incubation requirements
>> ================================
>> Once something becomes an integrated project, it's important that they
>> are able to run in the gate.
>>
>> Both devstack and devstack-gate now support hooks, so with a couple of
>> days of work any project in stackforge could build a gate job which
>> sets up a devstack of their configuration, including their code,
>> running some project specific test they feel is appropriate to ensure
>> they could run in the gate environment.
>>
>> This would ensure an incubated project works with OpenStack global
>> requirements, or if it requires something new, that's known very
>> clearly before incubation.
>>
>>
>> Proposed Graduation requirements
>> ================================
>> All integrated projects should be in the integrated gate, as this is
>> the only way we provably know that they can all work together, at the
>> same level of requirements, in a consistent way.
>>
>> During incubation landing appropriate tests in Tempest is fair
>> game. So the expectation would be that once a project is incubated
>> they would be able to land tests in tempest. Before integrated we'd
>> need to ensure the project had tests which could take part in the
>> integrated gate, so as soon as a project is voted integrated, it has
>> some working integrated gate tests. (Note: there is actually a
>> symmetric complexity here, to be worked out later).
>>
>>
>> Proposed Stable Release requirements
>> ====================================
>> We have this automatic transition that happens when a project that's
>> integrated for a release, actually releases as part of
>> that. I.e. Trove and Icehouse. There is no additional TC decision
>> about whether or not Trove is part of the stable release, once
>> integrated, it just is. Nothing that it does over that cycle will kick
>> it out of the stable release. This is one of the reasons it needs to
>> be in the integrated gate **before** graduation.
>>
>> Additionally, upgrade path is critically important to our users, and
>> the number one piece of feedback we received from the User Survey. It
>> was also important enough to our developers that it was scattered all
>> over the Icehouse Design Summit. All integrated projects should be
>> included in upgrade testing the moment they are in a stable
>> release. (ex: when Icehouse is released, Trove should be in master
>> grenade, and upgrade testing from Icehouse -> master for the J cycle
>> from day one).
>>
>>
>>
>> TL;DR
>> =====
>> - To get to incubated you must have a devstack-gate job
>> - To get to integrated you must have useful tests in the integrated
>>   gate
>> - To get into the stable/release you must have upgrade testing enabled
>>
>> Raised Questions
>> ================
>> - what about existing incubated projects, what would be their time
>>   frame to get with this new program
>> - what about existing integrated projects that currently don't exist
>>   with either an upgrade or gate story?
>> - what about an upgrade deprecation path (i.e. nova-network =>
>>   neutron, nova-baremetal => ironic)
>>
>>
>> 1.
>> http://hackstack.org/x/blog/2013/09/05/openstack-seven-layer-dip-as-a-service/
>>    (the layers pattern is useful from a testing requirements model)
>>
>>
>> --
>> Sean Dague
>> http://dague.net
>>
>>
>> _______________________________________________
>> OpenStack-TC mailing list
>> OpenStack-TC at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-tc
>>
> 
> 
>