[openstack-dev] My failed attempt to speed up nova tests

Matthew Sherborne msherborne at gmail.com
Sun May 12 23:34:21 UTC 2013


Until the wheel thing is available, this might help in making .venv
creation quicker:

mkdir -p ~/.pip/cache
echo export PIP_DOWNLOAD_CACHE=~/.pip/cache >> ~/.bashrc

Then it makes a nice cache and every time you create a new .venv it doesn't
need to download again.

Kind Regards,
Matthew Sherborne





On Tue, May 7, 2013 at 1:51 AM, Monty Taylor <mordred at inaugust.com> wrote:

>
>
> On 05/04/2013 07:59 PM, Matthew Sherborne wrote:
> > So I tried to speed up the running of my test suite and failed.
> >
> > I thought I'd share my experience in case someone else in the future
> > googles before
> >  trying the same thing, or in case some genius out there sees some fatal
> > flaw in my plan.
> >
> > Here's what I did:
> >
> > I'm on Ubuntu Raring. I have both gcc and clang compilers installed.
> >
> > I did some experiments with a c++ test program and found that clang with
> > the -O3 optimization flag creates a faster program than gcc with -O6.
> >
> > So I thought, I'll compile my own python and venv with python and clang
> > and -O3.
> >
> > So I compiled my python from source with clang and -O3 and installed it
> > in ~/python.
> >
> > I deleted my venv for nova and, rebuilt it all under the new python.
> >
> > Here are the *results* of: time ./run_tests.sh
> >
> > *standard python:*
> >
> > real    7m4.077s
> > user    21m0.080s
> > sys 0m24.596s
> >
> > *with clang compiled python:*
> >
> > real    8m11.396s
> > user    24m28.824s
> > sys 0m25.408s
> >
> > I tried it a few times and got similar results. So it seems that the
> > stock ubuntu python is faster than my cmake -O3 compiled one.
> >
> > All comments and suggestions welcomed.
>
> Nice experiment! I do love seeing actual numbers.
>
> BTW - we've got a back-burner project to add support for the new python
> wheel file format to our pypi mirror, and then to build wheels of things
> when we pull them down from upstream. While this will do nothing to
> speed up the actual test runs, it should make virtualenv creation WAY
> quicker, since we won't have to continually re-compile libxml...
>
> thanks for experimenting with that!
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130513/7d543813/attachment.html>


More information about the OpenStack-dev mailing list