Open Stack

On 07/26/2013 04:59 PM, Joe Gordon wrote:
> 
> 
> 
> On Fri, Jul 26, 2013 at 11:34 AM, Jay Pipes <jaypipes at gmail.com
> <mailto:jaypipes at gmail.com>> wrote:
> 
>     On 07/25/2013 04:21 PM, Joe Gordon wrote:
> 
>         Hi All,
> 
>         We have recently hit some performance issues with nova-network.  It
>         turns out the root cause of this was we do roughly 20
>         rootwrapped shell
>         commands, many inside of global locks.
>         (https://bugs.launchpad.net/__oslo/+bug/1199433
>         <https://bugs.launchpad.net/oslo/+bug/1199433>)
> 
>         It turns out starting python itself, has a fairly significant
>         overhead
>         when compared to the run time of many of the binary commands we
>         execute.
> 
>         For example:
> 
>         $ time python -c "print 'test'"
>         test
> 
>         real0m0.023s
>         user0m0.016s
>         sys0m0.004s
> 
> 
>         $ time ip a
>         <...>
> 
>         real0m0.003s
>         user0m0.000s
>         sys0m0.000s
> 
> 
>         While we have removed the extra overhead of using entry points,
>         we are
>         now hitting the overhead of just shelling out to python.
> 
> 
>     Hey Joe,
> 
>     Just ask a question of you personally, off list... I was curious
>     about the above statement about entry points. Could you elaborate a
>     bit there? I thought OpenStack had moved *to* using entry points
>     with stevedore?
> 
> 
> Turns out this wasn't off the list.  So I will answer on the list.
> 
> I am not sure about how stevadore fits into this, but the issue is
> related to 'import pkg_resources' and how the binaries are built from
> the 'cmd' files.   It looks like nova doesn't use stevadore for that
> (yet?). It turns out pkg_resources scans your entire python environment
> in part to make it possible to have multiple versions of packages
> installed, but it looks like pip doesn't support that anyway so it just
> slows things down and provides us no benefit. 
> 
> You can see the issue by going into devstack and running "time python -c
> 'import pkg_resouces'"
> 
> This thread may shine more light on what we
> saw: http://mail.python.org/pipermail/distutils-sig/2013-July/021924.html
> 
> Not sure if that helps.
>  
> 
> 
>     Did this change recently?

Lemme expand real quick.

We're still declaring console_script entry_points in setup.cfg and the
interface is exactly the same. However, we're now causing the generated
script to do a direct import rather than a pkg_resources look up. In the
context of the console_script, the pkg_resources look up is overhead in
support of multiple parallel installs, but since we're a pip shop, those
are not available to us anyway, so it's all cost for no benefit.

The switch to direct import in the generated script is in pbr trunk
(which means it's what devstack is using now) We should be making a new
pbr real-soon-now.

Monty