Open Stack

On Sep 26, 2014, at 2:27 PM, Sean Dague <sean at dague.net> wrote:

> As we've been talking about the test disaggregation the hamster wheels
> in the back of my brain have been churning on the library testing
> problem I think we currently have. Namely, openstack components mostly
> don't rely on released library versions, they rely on git master of
> them. Right now olso and many clients are open masters (their
> stable/juno versions are out), but we've still not cut RCs on servers.
> So we're actually going to have a time rewind event as we start cutting
> stables.
> 
> We did this as a reaction to the fact that library releases were often
> cratering the world. However, I think the current pattern leads us into
> a much more dangerous world where basically the requirements.txt is invalid.
> 
> So here is the particular unwind that I think would be useful here:
> 
> 1) Change setup_library in devstack to be able to either setup the
> library from git or install via pip. This would apply to all libraries
> we are installing from oslo, the python clients, stackforge, etc.
> Provide a mechanism to specify LIBRARIES_FROM_GIT (or something) so that
> you can selectively decide to use libraries from git for development
> purposes.
> 
> 2) Default devstack to use pip released versions.

These first 2 suggestions seem good. It makes our test configuration more accurate, and frees the Oslo team to have our dev cycle a little out of phase without worrying about breaking things. It means app developers will have to wait for fixes and features longer than they would otherwise, but Thierry has improved the automation for release tracking so we could release more often.

> 
> 3) Change the job definition on the libraries to test against devstack
> in check, not in gate. The library teams can decide if they want their
> forward testing to be voting or not, but this is basically sniff testing
> that when they release a new library they won't ruin the world for
> everyone else.

The other suggestions only work if we also have some job that does still use the source version of the libraries. A voting check job is good. Why would we not also do the test in the gate?

> 
> 4) If a ruin the world event happens, figure out how to prevent that
> kind of event in local project testing, unit or functional. Basically an
> unknown contract was broken. We should bring that contract back into the
> project itself, or yell at the consuming project about why they were
> using code in a crazy pants way.
> 
> Additionally, I'd like us to consider: No more alpha libraries. The
> moment we've bumped global requirements in projects we've actually
> released these libraries to production, as people are CDing the servers.
> We should just be honest about that and just give things a real version.
> Version numbers are cheap.

Anyone following trunk is already expecting some pain, but the folks that follow the stable branches do it because we call them stable. We have been using alpha versions as a way to avoid throwing development versions of libraries into the stable branch test and production environments before they have had some runtime against master. It’s not so much an indication of the quality of the content of the package as a hack taking advantage of pip behaviors that don’t install pre-release packages unless specifically told do so, and a signal to distros that they don’t necessarily have to package those releases (I don’t know if they’re receiving that signal or not). If we stop using alpha version numbers, we need some other way to avoid having development releases end up in the stable test environments. Or to say explicitly that it’s OK if they do, I guess. 

I would also like to hear some from the distro packaging folks about what it would mean for us to be releasing regular version libraries frequently (expect at least one release every week for the entire cycle). Would you package all of those, or would you wait until closer to the end of the cycle and package the “final” versions?

Doug

> 
> 	-Sean
> 
> -- 
> Sean Dague
> http://dague.net
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev