[openstack-dev] [Oslo][Neutron] Fork() safety and oslo.messaging

Mehdi Abaakouk sileht at sileht.net
Tue Nov 25 16:16:16 UTC 2014

Hash: SHA256

> Mmmm... I don't think it's that clear (re: an application issue).  I
> mean, yes - the application is doing the os.fork() at the 'wrong'
> time, but where is this made clear in the oslo.messaging API
> documentation?
> I think this is the real issue here:  what is the "official" guidance
> for using os.fork() and its interaction with oslo libraries?
> In the case of oslo.messaging, I can't find any mention of os.fork()
> in the API docs (I may have missed it - please correct me if so).
> That would imply - at least to me - that there is _no_ restrictions of
> using os.fork() together with oslo.messaging.

Yes, I agree we should add a note on that in oslo.messaging (and perhaps 
in oslo.db too).

And also the os.fork() is done by the service.ProcessLauncher of 
and it's not (yet) documented. But once oslo.service library will be 
released, it will.

> But in the case of qpid, that is definitely _not_ the case.
> The legacy qpid driver - impl_qpid - imports a 3rd party library, the
> qpid.messaging API.   This library uses threading.Thread internally,
> we (consumers of this library) have no control over how that thread is
> managed.  So for impl_qpid, os.fork()'ing after the driver is loaded
> can't be guaranteed to work.   In fact, I'd say os.fork() and
> impl_qpid will not work - full stop.

Yes, I have tried it, and I have catch what happen and I can confirm 
that too now, unfortunately :( And this can occurs with any driver if 
the 3rd party library
doesn't work when we use os.fork()

>> For the amqp1 driver case, I think this is the same things, it seems 
>> to do lazy creation of the connection too.
> We have more flexibility here, since the driver directly controls when
> the thread is spawned.  But the very fact that the thread is used
> places a restriction on how oslo.messaging and os.fork() can be used
> together, which isn't made clear in the documentation for the library.
> I'm not familiar with the rabbit driver - I've seen some patch for
> heatbeating in rabbit introduce threading, so there may also be an
> implication there as well.

Yes, we need to check that.

>> Personally, I don't like this API, because the behavior difference 
>> between
>> '__init__' and 'start' is too implicit.
> That's true, but I'd say that the problem of implicitness re:
> os.fork() needs to be clarified at the library level as well.

I agree.

I will write the documentation patch for oslo.messaging.


- ---
Mehdi Abaakouk
mail: sileht at sileht.net
irc: sileht
Version: OpenPGP.js v.1.20131017
Comment: http://openpgpjs.org


More information about the OpenStack-dev mailing list