Open Stack

On Oct 10, 2014, at 12:52 PM, Ihar Hrachyshka <ihrachys at redhat.com> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
> 
> On 10/10/14 20:21, Joshua Harlow wrote:
>> On Oct 10, 2014, at 3:13 AM, Ihar Hrachyshka <ihrachys at redhat.com>
>> wrote:
>> 
>> On 09/10/14 21:29, Mike Bayer wrote:
>>>>> So so far, everyone seems really positive and psyched about
>>>>> the proposal.
>>>>> 
>>>>> It looks like providing some options for how to use would be
>>>>> best, that is provide decorators and context managers.
>>>>> 
>>>>> Now the thing with the context manager, it can be as I
>>>>> stated:
>>>>> 
>>>>> with sql.reader() as session:
>>>>> 
>>>>> or we can even have that accept the “context”:
>>>>> 
>>>>> with sql.reader(context) as session:
>>>>> 
>>>>> The latter again avoids having to use thread locals.
>>>>> 
>>>>> But can I get a feel for how comfortable we are with using
>>>>> thread local storage to implement this feature?   I had
>>>>> anticipated people wouldn’t like it because it’s kind of a
>>>>> “global” object, even though it will be hidden behind this
>>>>> facade (of course CONF is global as is sys.modules, and I
>>>>> think it is fine).     If I just use a tlocal, this whole
>>>>> thing is pretty simple.
>> 
>> Won't the approach conflict with eventlet consumers that for some 
>> reason do not patch thread module, or do not patch it early enough?
>> I guess in that case we may end up with mixed contexts.
>> 
>>> Eck, this is not something people should be doing (not patching
>>> the thread module).
>> 
>>> Example for why @
>>> https://gist.github.com/harlowja/9c35e443dfa136a4f965 (run that
>>> and see your program lock up).
>> 
>>> Once you accept/use eventlet, u shouldn't really be accepted half
>>> of it, otherwise there be dragons there; especially since
>>> openstack doesn't control what external library code does inside
>>> those libraries (and rightfully so it shouldn't), and if any
>>> library does anything like that gist code, the whole application
>>> will lock up...
> 
> Real life sucks. In Neutron, we have at least two files that patch
> only some modules (though all of them patch threading). [Not that I
> mean that those files should not patch the whole library set; I have
> no defined view on the matter.]
> 

Yup it does, and once u accept eventlet it starts to become hard to get off it...

It's 'viral', and for better or worse openstack has got the virus... So that means everything that openstack touches/uses also needs to be compatible with that virus... It's to bad guido just didn't accept eventlet/greenlet into mainline python, that would of made our job easier (and then added a python option or something like `python -g` that turns on greenthreading by default when the interpreter is started up)...

Thats the crappy part with anything that monkey patches all the things, its not so easy to accept half of a monkey...

> Though I agree that consumers with some modules unpatched are not
> something that should be seriosly considered, we're still left with
> consumers that do not patch their libraries early enough. We had an
> issue with tlocal when porting Neutron to oslo.messaging that resulted
> in the following patch: https://review.openstack.org/#/c/96791/ The
> issue showed up in very weird way hard to debug, so if possible, I
> would try to avoid similar situations with oslo.db.

I'm unsure on this one, I don't think we should be making oslo.* that succumbs to the same eventlet virus just because openstack projects aren't correctly using eventlet. The same bug that was found in oslo.messaging could of been any third part library that wasn't imported in the right order in all honesty. So it seems like we should fix the projects (as in that review) instead of making the oslo.* libraries less future compatible... If eventlet/greenlet got merged into mainline python then I'd say probably say the reverse, but it didn't.

> 
> /Ihar
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
> 
> iQEcBAEBCgAGBQJUODkZAAoJEC5aWaUY1u57QrcIANLtMi1jRrfHnwV2xnwYGPOl
> QwZWDG6i/cdnzqrwGz+0chqRxu4KWZB5gAfMR4/ralHvGGbaa1vGp9wtzsfIJTB1
> RuyE7XKXUX4rU3WoAOB7F3uF1WzFpI8ourunBG4OCE0t0YT89z9mCLfOTZNNKGxH
> OE89n4pnyMd3GXGtoxVINyA6pqQotYuHKG6o6LbhmTZ1UoL3P3rO+Onb1Ma2BF6Z
> VO84smdMnOC3yVtv4TGwrW5iBoU3RwOqxooOg4g5Jam6D++kzCKrPT9wdY7TNe+C
> A6IvMlad0RbygkhrLyWIWXaqCAg5VzzDvk4z6dywVkmoWS7V861mjiV4unbr0X8=
> =5ovh
> -----END PGP SIGNATURE-----
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev