[openstack-dev] Should we be using subprocess32?

Doug Hellmann doug at doughellmann.com
Fri Sep 15 15:08:39 UTC 2017


Excerpts from Tony Breeds's message of 2017-09-14 16:18:35 -0600:
> On Tue, Sep 12, 2017 at 11:29:22AM -0700, Joshua Harlow wrote:
> > Hi folks,
> > 
> > I know there is a bunch of usage of subprocess in openstack and especially
> > since there is heavy usage of python 2.7 it made me wonder if we should try
> > to move to subprocess32 to avoid some of the bugs that seem to exist (maybe
> > distributors backported them?):
> > 
> > For example a major one (seems to be):
> > 
> > - https://github.com/google/python-subprocess32/commit/6ef1fea55
> > 
> > """Popen.wait() is now thread safe so that multiple
> > 
> >   threads may be calling wait() or poll() on a Popen instance at the same
> > time
> >   without losing the Popen.returncode value.
> > """
> > 
> > That one concerns me slightly, because I know that certain openstack
> > projects do use threads (and not eventlet monkey-patched green-thread
> > hybrids).
> > 
> > TLDR; should we (could we?) switch?
> 
> We could.  It wouldn't be hard to propose a change to the requirements
> repo and that could be used to test what breaks.
> 
> As to should we I'm not convinced.  It does give us a slightly more
> modern subprocess module but it hasn't been updated in nearly 2 years.
> I get that it's a backport from 3.3 which isn't getting updated but ...
> 
> Also it means adding something like:
> if os.name == 'posix' and sys.version_info[0] < 3:
>     import subprocess32 as subprocess
> else:
>     import subprocess
> 
> All over the place which isn't so great.
> 
> So overall I'm not certain it's worth it.
> 
> Yours Tony.

We might get more value by continuing the migration to python 3 so we
can drop python 2 support.

Doug



More information about the OpenStack-dev mailing list