Open Stack

On 01/03/2014 08:27 PM, Robert Collins wrote:
> On 4 January 2014 08:44, Doug Hellmann <doug.hellmann at dreamhost.com> wrote:
>> It seems safer to gate changes to libraries against the apps' trunk (to
>> avoid making backwards-incompatible changes), and then gate changes to the
>> apps against the released libraries (to ensure they work with something
>> available to be packaged by the distros). There are lots and lots of version
>> numbers available to us, so I see no problem with releasing new versions of
>> libraries frequently.

So we used to do that the apps against release libraries. And the result 
was more and more full day gate breaks. We did 2 consecutive ones in 2 
weeks.

Basically, once you get to be a certain level of coupled in OpenStack we 
can no longer let you manage your own requirements file. We need a 
global lever on it. Because people were doing it wrong, and slowly (we 
could go through specific examples about how bad this was. This was a 
top issue at nearly every summit I'd been at going back to Essex.

Some reading from the break times:
  * http://lists.openstack.org/pipermail/openstack-dev/2013-July/011660.html
  * http://lists.openstack.org/pipermail/openstack-dev/2013-July/011708.html
  * http://lists.openstack.org/pipermail/openstack-dev/2013-July/012342.html

(It was about 14 days to resolve the python client issue, there was a 
django issue around the same time that never made it to the list, as we 
did it all under fire in IRC)

And we have a solution now. Which is one list of requirements that we 
can test everything with, that we can propose requirements updates 
speculatively, and see what works and what doesn't. And *after* we know 
they work, we propose the changes back into the projects, now automatically.

> I do see the issue Sean is pointing at, which is that we have to fix
> the libraries first and then the things that use them. OTOH thats
> normal in the software world, I don't see anything unique about it.

Well, as the person that normally gets stuck figuring this out when .eu 
has been gate blocked for a day, and I'm one of the first people up on 
the east coast, I find the normal state of affairs unsatisfying. :)

I also think that what we are basically dealing with is the classical 
N^2 comms problem. With N git trees that we need to all get working 
together, this gets exponentially more difficult over time. Which is why 
we created the integrated gate and the global requirements lever.

Another solution would be reduce the number of OpenStack git trees to 
make N^2 more manageable, and let us with single commits affect multiple 
components. But that's not the direction we've taken.

	-Sean

-- 
Sean Dague
Samsung Research America
sean at dague.net / sean.dague at samsung.com
http://dague.net