Open Stack

On Sat, Jan 4, 2014 at 8:02 AM, Sean Dague <sean at dague.net> wrote:

> On 01/03/2014 08:27 PM, Robert Collins wrote:
>
>> On 4 January 2014 08:44, Doug Hellmann <doug.hellmann at dreamhost.com>
>> wrote:
>>
>>> It seems safer to gate changes to libraries against the apps' trunk (to
>>> avoid making backwards-incompatible changes), and then gate changes to
>>> the
>>> apps against the released libraries (to ensure they work with something
>>> available to be packaged by the distros). There are lots and lots of
>>> version
>>> numbers available to us, so I see no problem with releasing new versions
>>> of
>>> libraries frequently.
>>>
>>
> So we used to do that the apps against release libraries. And the result
> was more and more full day gate breaks. We did 2 consecutive ones in 2
> weeks.
>
> Basically, once you get to be a certain level of coupled in OpenStack we
> can no longer let you manage your own requirements file. We need a global
> lever on it. Because people were doing it wrong, and slowly (we could go
> through specific examples about how bad this was. This was a top issue at
> nearly every summit I'd been at going back to Essex.
>
> Some reading from the break times:
>  * http://lists.openstack.org/pipermail/openstack-dev/2013-
> July/011660.html
>  * http://lists.openstack.org/pipermail/openstack-dev/2013-
> July/011708.html
>  * http://lists.openstack.org/pipermail/openstack-dev/2013-
> July/012342.html
>
> (It was about 14 days to resolve the python client issue, there was a
> django issue around the same time that never made it to the list, as we did
> it all under fire in IRC)
>
> And we have a solution now. Which is one list of requirements that we can
> test everything with, that we can propose requirements updates
> speculatively, and see what works and what doesn't. And *after* we know
> they work, we propose the changes back into the projects, now
> automatically.


>  I do see the issue Sean is pointing at, which is that we have to fix
>> the libraries first and then the things that use them. OTOH thats
>> normal in the software world, I don't see anything unique about it.
>>
>
> Well, as the person that normally gets stuck figuring this out when .eu
> has been gate blocked for a day, and I'm one of the first people up on the
> east coast, I find the normal state of affairs unsatisfying. :)
>

I appreciate you going through the details with me.

As I said at the beginning, I'm not opposed to bringing taskflow into the
oslo program at all -- and we can still do that, if Josh wants to,
retaining the existing core review team, permissions, etc. -- but since we
plan to release more libraries over time I want to make sure that we're set
up to handle cases like this where we have access to the developers and
source for a library, but don't call it an "openstack" library.


>
> I also think that what we are basically dealing with is the classical N^2
> comms problem. With N git trees that we need to all get working together,
> this gets exponentially more difficult over time. Which is why we created
> the integrated gate and the global requirements lever.
>
> Another solution would be reduce the number of OpenStack git trees to make
> N^2 more manageable, and let us with single commits affect multiple
> components. But that's not the direction we've taken.


The global requirements syncing seems to have fixed the issue for apps,
although it just occurred to me that I'm not sure we check that the
requirements lists are the same when we cut a release. Do we do that
already?

Doug



>
>
>         -Sean
>
> --
> Sean Dague
> Samsung Research America
> sean at dague.net / sean.dague at samsung.com
> http://dague.net
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140106/0d6ef6d2/attachment.html>