Open Stack

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

I have a few remarks (mostly around our enforcement of grenade
testing, since there is little we can do once the project is
"integrated"), but I don't want to start a larger discussion on the
- -tc list... Could you repost this to -dev first ?

(rationale: we must be careful not to have "closed" discussions on the
- -tc list so that the TC is not seen as a kabbale / ivory tower -- so
the -tc list should be limited to TC meeting organization
administrativia and posting pointers to other threads of interest to
TC members)

Thanks,

Sean Dague wrote:
> There were a few chats at summit about this, mostly on the infra / 
> devstack / qa side of the house. Consider the following a straw man
> to explain the current state of the world, and what I think a
> starting point for discussion would be. I call out projects by name
> here, not to make fun of them, but that I think concrete examples
> bring the point home much more than abstracts (at least for me).
> 
> This is looking at raising the bar quite a bit along the way.
> However, as someone that spends a lot of time trying to keep the
> whole ball of wax holding together, and is spending a lot of time
> retroactively trying to get projects into our integrated gate (and
> huge pain to everyone, as their gate commits get bounced by racey
> projects), I think we really need to up this bar if we want a 20
> integrated project version of OpenStack to hold together.
> 
> ===================================================== Proposed new
> Incubation and Graduation Requirements 
> =====================================================
> 
> 
> The Current State of the World ============================== 
> OpenStack is currently 9 integrated projects in Havana, 10 once we
> get to Icehouse (+Trove).
> 
> Heat added tests to our integrated gate during the Havana cycle, 
> though the deeper guest testing is not part of our gate.
> Ceilometer has been integrated for 2 releases, and doesn't have
> integrated gate testing. Trove is just now exploring how they'd get
> into the gate for Icehouse release.
> 
> Upgrade testing situation is worse. Only 5 projects currently are
> part of upgrade testing: Nova, Cinder, Swift, Glance, Keystone. The
> rest are not set up in that model at all, so we have no
> infrastructure to ensure the other projects don't break upgrades of
> themselves.
> 
> Heat, Trove, and Savana represent an interesting challenge from 
> current gate models in that real validation requires something
> beyond a trivial guest, as they care about the contents inside
> compute resources. This desire is likely to grow with other "Layer
> 4" projects coming into the OpenStack ecosystem [1].
> 
> Ironic as an incubated project provides an even different set of 
> challenges, as not only do we not have a gating approach, but we
> also really should have an approach to test an upgrade from
> nova-baremetal => ironic to ensure there is a migration path for
> people in the future.
> 
> Ceilometer once relied on a version of MongoDB that worked in the 
> gate, but they were never gating, so now they rely on a version of 
> MongoDB that doesn't work in the gate. They made a technical
> decision to upgrade requirements with no gate feedback because they
> weren't actually integration testing on a regular basis.
> 
> 
> Proposed Incubation requirements ================================ 
> Once something becomes an integrated project, it's important that
> they are able to run in the gate.
> 
> Both devstack and devstack-gate now support hooks, so with a couple
> of days of work any project in stackforge could build a gate job
> which sets up a devstack of their configuration, including their
> code, running some project specific test they feel is appropriate
> to ensure they could run in the gate environment.
> 
> This would ensure an incubated project works with OpenStack global 
> requirements, or if it requires something new, that's known very 
> clearly before incubation.
> 
> 
> Proposed Graduation requirements ================================ 
> All integrated projects should be in the integrated gate, as this
> is the only way we provably know that they can all work together,
> at the same level of requirements, in a consistent way.
> 
> During incubation landing appropriate tests in Tempest is fair 
> game. So the expectation would be that once a project is incubated 
> they would be able to land tests in tempest. Before integrated
> we'd need to ensure the project had tests which could take part in
> the integrated gate, so as soon as a project is voted integrated,
> it has some working integrated gate tests. (Note: there is actually
> a symmetric complexity here, to be worked out later).
> 
> 
> Proposed Stable Release requirements 
> ==================================== We have this automatic
> transition that happens when a project that's integrated for a
> release, actually releases as part of that. I.e. Trove and
> Icehouse. There is no additional TC decision about whether or not
> Trove is part of the stable release, once integrated, it just is.
> Nothing that it does over that cycle will kick it out of the stable
> release. This is one of the reasons it needs to be in the
> integrated gate **before** graduation.
> 
> Additionally, upgrade path is critically important to our users,
> and the number one piece of feedback we received from the User
> Survey. It was also important enough to our developers that it was
> scattered all over the Icehouse Design Summit. All integrated
> projects should be included in upgrade testing the moment they are
> in a stable release. (ex: when Icehouse is released, Trove should
> be in master grenade, and upgrade testing from Icehouse -> master
> for the J cycle from day one).
> 
> 
> 
> TL;DR ===== - To get to incubated you must have a devstack-gate
> job - To get to integrated you must have useful tests in the
> integrated gate - To get into the stable/release you must have
> upgrade testing enabled
> 
> Raised Questions ================ - what about existing incubated
> projects, what would be their time frame to get with this new
> program - what about existing integrated projects that currently
> don't exist with either an upgrade or gate story? - what about an
> upgrade deprecation path (i.e. nova-network => neutron,
> nova-baremetal => ironic)
> 
> 
> 1. 
> http://hackstack.org/x/blog/2013/09/05/openstack-seven-layer-dip-as-a-service/
>
> 
(the layers pattern is useful from a testing requirements model)
> 
> 
> 
> 
> _______________________________________________ OpenStack-TC
> mailing list OpenStack-TC at lists.openstack.org 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-tc
> 


- -- 
Thierry Carrez (ttx)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJSg1ESAAoJEFB6+JAlsQQjn4YQAJwf8xIBcZHqPFAuqyla1RsT
jIRmCvG7xOmUp4tcr5phUx8aJomnPCu00GEmt3vpyhQnslE8q8BqFgzyMYCFiqc8
vYx6byvXBG8xr4E5/jeTYzIwChnxZg1t9svlIHH23jHbT0/8lI8AnV465lM0urUX
XfEWVnGE8TB40MwQV39wrqnJJ4iLlplPLKSI5dqddY3r9xpJwO2MBjHpvf5rL0qW
vJKnhgRGV2S+pUiWW+L9i5cHYp1AHkiCIoEJOUxcBQ2eh/46xy7kaiuTUzRTNM6+
prME3/mpEzbcK1IjURCYQVTAsCzihatqiAYDAq7MoqHUOem04FP6tC8Kx3UpaF1J
1xZ0SL4TP8m/Q9RYOysOvHLy94D8bSKe6smcL9Eh7f4lpvW1lx4vSYWv/khcFzz+
QJFr2YHN3MfZC0TCBN1A2h7Dy28z2GP6anl9jlZdZ1Y1CwdG7H+Cp7lxnpJGTDII
R/8hhnXGo//TWNSFfTvEreUNmlwpfS0bCwrBAG+sBS8LKY1uhMYbcrP2Nx98H4fS
kC/Lm5DDBi3Dk+243QU4UO4d8RBHZd/BIFWco7wMiIjzPEYqqBn71RxcGn8sFTis
DGSDMxArYC0sFdB0Si3F4iJHUYKEpOPgYfGbvRouxamruQ1y9bjvSbH07qfiKjU9
9wqET0YI0t3f8bkfvlfn
=4+dw
-----END PGP SIGNATURE-----