Open Stack

Since the library model is what most everyone else uses outside of
openstack (I assume?) what can we do to get there so that this model works
as well?

Expanding dependencies recursively seems like it could help? This could
then detect transitive dependency issues (and doesn't seem so hard to do).

I like the idea of 'gate on all the things' (that I've heard come up
before) but I don't know if its possible? If we hooked into the pypi
upload 'stream' it would seem like we could automatically trigger
openstack builds when a known dependency (or dependency of a
dependency...) is uploaded (maybe?). That would be pretty neat.

In general it really seems like having more libraries, not less is ideal
(making us solve this issue whether we want to or not really). As
libraries can then be used outside of openstack (taskflow is already being
used elsewhere also), libraries create well-defined apis and boundaries
between programs (...). I know they also create this dependency hell
problem (anvil has been hitting these same issues for a while to). But I
think we can figure out a solution that fits both worlds, the world of
things we can gate on and the world of things we can't (3rd party
libraries that aren't under openstack control). Taskflow is in-between
those worlds, so it allows us to explore how to make both of those worlds
work.

-Josh

On 1/3/14, 2:54 PM, "Sean Dague" <sean at dague.net> wrote:

>On 01/03/2014 05:10 PM, Ivan Melnikov wrote:
>> On 04.01.2014 01:29, Sean Dague wrote:
>>> On 01/03/2014 04:17 PM, Doug Hellmann wrote:
>> [...]
>>>> That's what made me think of the solution. But isn't setuptools in
>>>>fact
>>>> telling us that somehow the versions of things we expected to have
>>>> installed are no longer installed and so something *is* broken (even
>>>>if
>>>> the versions of the installed libraries work together).
>>>
>>> It actually tells us that a human, somewhere, decided that their
>>> software did not work with some combination of other software, and that
>>> we are no longer able to correct their mistaken assumptions.
>> [...]
>>
>> But sometimes humans are not wrong. For example, no released TaskFlow
>> version really works with SQLAlchemy >= 0.8 -- that was fixed only
>> recently (https://review.openstack.org/#/c/62661/).
>>
>> I consider requirements to be part of the code, so if they are too
>> strict (or too broad), that must be fixed, in a usual opensource way:
>> via filing bug report, suggesting patch and so on.
>>
>> Requirements should reflect reality, especially for libraries that are
>> intended to be useful outside of OpenStack, too. For example, current
>> TaskFlow requirement for SQLAlchemy is too strict, so we'll fix that,
>> and release new version with that fix.
>
>Well part of the problem is because of it being a dependency of a
>dependency, our global requirements process couldn't explore whether or
>not it functioned in a real environment.
>
>Which brings us back to the idea of making this a project that works in
>the integrated gate.
>
>It's also kind of problematic that apparently the introduction of
>taskflow actually caused a regression from Havana (which the distros
>managed to make work with sqla 0.8 even though it wasn't in global
>requirements, but this apparently would have broken).
>
>And the next question is when is a sqla 0.9 compatible version going to
>be out there? Because we can't even explore allowing openstack to use
>sqla 0.9 until taskflow does, again because it's a blocking requirement.
>
>It's exactly these kind of lock step problems that we avoid with the
>rest of openstack by making it an integrated gate with global
>requirements sync. But that we can't really handle with the library model.
>
>	-Sean
>
>-- 
>Sean Dague
>Samsung Research America
>sean at dague.net / sean.dague at samsung.com
>http://dague.net
>
>_______________________________________________
>OpenStack-dev mailing list
>OpenStack-dev at lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev