[openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

Joshua Harlow harlowja at yahoo-inc.com
Sun Jan 5 16:09:20 UTC 2014


With regards to the futures module it should just work fine with the packaging of https://pypi.python.org/pypi/futures which is a backport of the 3.2 concurrent futures package into 2.6,2.7, so that's the package there.

Feel free to bug me on irc if u want any other help with dependencies, the entrypoint failures shouldn't happen (possibly due to this). If u need help just bug "harlowja" on irc or jump in "#openstack-state-management" where others can help to...

Sent from my really tiny device...

> On Jan 4, 2014, at 7:26 AM, "Thomas Goirand" <zigo at debian.org> wrote:
> 
> Sean,
> 
> Before everything, I'd like to thank you for insisting in making the
> transition to SQLA 0.8.x.
> 
> Since it has been uploaded to Sid, this SQLA <0.7.99 has been without
> any doubt the biggest reoccurring pain in the but with the packaging of
> OpenStack. Without people like you, insisting again and again, I would
> have loose hope that progress could happen in OpenStack! So thanks
> again, Sean.
> 
>> On 01/03/2014 11:26 PM, Sean Dague wrote:
>> Given that sqla 0.9 just came out, I wanted to explore, again, what the
>> state of the world was with sqla 0.8 (especially given that Ubuntu and
>> Red Hat are both shipping 0.8 in their OpenStack bundles) -
>> https://review.openstack.org/#/c/64831/
>> 
>> The answer is not great. But more importantly, the answer is actually
>> worse than the last time we checked, which I think gives this some
>> urgency to move forward.
> 
> For me, it's been urgent since the 6th of July...
> 
> SQLA 0.8.2 was uploaded to Sid on the 6th of July. Since then, I've been
> bugging everyone on this list about it, explaining that I have no choice
> but to have Debian packages to support it (since I upload in Sid, and
> Sid has SQL 0.8.x). It seems I still haven't been heard.
> 
> Now, we're 6 months after that, and after the release of Havana which
> happened more than 3 months after this, and after everything was fixed
> in all core packages (the last one was heat, at the end of August). So,
> in January 2014, I'm still fixing manually most requirements.txt, which
> still advertize for support of only SQL <0.7.99. I currently have
> fixes-SQLAlchemy-requirement.patch in Cinder, Neutron, and Nova (for
> Havana), just to allow it and stop the software to break because it was
> decided it is the case, even though they perfectly work with SQLA 0.8.
> Some other project have SQLAlchemy>=0.7.8,<=0.7.99 in the
> requirements.txt, but do not break as badly as these 3 just because of
> the bad declaration that doesn't reflect the reality (that's the case
> for Heat Keystone and Glance, for example).
> 
> Something is going extremely wrong here. And seeing the actual result of
> the SQLA transition, I am really leaning toward thinking we have a
> process problem. What I believe is wrong, is that instead of having
> project wide decisions imposing some constraints, we have leaf packages
> that do. Until absolutely all of OpenStack is ready and fixed, then
> there's no constraint applied. This is the thing that must change.
> 
> It shouldn't be this way. It should be from top to bottom. While I do
> understand that we do need the gate to be able to continue working with
> all projects at any given time, we still must find a solution so that
> this kind of 6 months transition never happens again. This really goes
> against the culture that we have inside OpenStack, and our common belief
> that things must be able to move fast.
> 
> On 01/04/2014 04:13 AM, Sean Dague wrote:>
>> Because of entry points any library that specifies any dependencies
>> that OpenStack components specify, at incompatible levels, means that
>> library effectively puts a hold on the rest of OpenStack and prevents
>> it from being able to move forward.
> 
> That's exactly the kind of *very bad* habits that needs to stop in
> OpenStack.
> 
>> On 01/04/2014 04:13 AM, Sean Dague wrote:
>> The only other option is that libraries we own (stackforge / oslo),
>> for condition to be included in global- requirements, *can never*
>> specify a maximum version of any dependency (and I really mean
>> never), and can never specify a minimum greater than current global
>> requirements.
> 
> PLEASE !!! Let's do this !!! :)
> 
>> On 01/04/2014 05:29 AM, Sean Dague wrote:
>> It actually tells us that a human, somewhere, decided that their
>> software did not work with some combination of other software
> 
> Often it's even worse. Sometimes, a human decide that, just it case, the
> software will declare itself incompatible with some non-existent future
> version of another software, even if there's no way to know.
> 
> We're even more into sci-fi when we see stuff like:
> 
> pbr>=0.5.21,<1
> 
> Monty, did you decide you would release 1.0 with lots of backward
> incompatibility? Has the topic been raised and I missed it??? I'm
> convinced this isn't the case (and let's pretend it isn't, just until
> the end of this message).
> 
> So, how does one know that the thing he's using in PBR will be the thing
> that will cause trouble later on? For a version which hasn't been
> released yet?!? Who's the person with such a looking glass, who can
> predict the future? I'd like to know as well some stuff in my personal
> future...
> 
> Please, let's stop this kind of non-sense and stop pretending we can
> tell if something is incompatible with a version of something else that
> doesn't even exist yet. It's hurting and slowing down the whole of
> OpenStack for no reason.
> 
>> One of the key problems is taskflow, which has an sqla pin, which breaks
>> all the cinder entry points. This was actually entirely the problem
>> global requirements was meant to address, but it really can't when there
>> are nested requirements like this, in stackforge projects that aren't
>> installing via git (so we can rewrite their requirements).
>> 
>> So why does taskflow have the SQLA pin? And if the answer is global
>> requirements, then taskflow can't be installing via pypi like it is now,
>> because it's by nature taking a wedge. So we need a real solution here
>> to un bind us, because right now it's actually impossible to upgrade
>> sqla in requirements because of taskflow.
> 
> A colleague and myself have been struggling with the packaging of
> taskflow. Whatever we do with its dependencies, there's some unit tests
> errors. One of them is that it can't find the "futures" module:
> 
> DistributionNotFound: futures>=2.1.3
> 
> That's weird because I have python-concurrent.futures 2.1.3-1~bpo70+1
> installed on my system.
> 
> There's also lots of:
> 
> RuntimeError: No 'taskflow.engines' driver found, looking for 'parallel'
> or
> RuntimeError: No 'taskflow.engines' driver found, looking for 'default'
> 
> which we can't figure out the reason (but probably related to the
> detection of futures missing...).
> 
> Though we do have python-concurrent.futures installed. Is this really
> the module that taskflow expects, or does it really want "futures" and
> not concurrent.futures? If it really wants "futures" then we have a
> problem, because python-futures (which isn't in Debian yet) would
> conflict with concurrent.futures (they would share the same files). I'd
> be very happy to have upstream point of view, and probably some help.
> 
> Cheers,
> 
> Thomas Goirand (zigo)
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



More information about the OpenStack-dev mailing list