Open Stack

Clark Boylan <cboylan at sapwetik.org> writes:

...

> I've since worked out a change that passes tempest using a global
> virtualenv installed devstack at
> https://review.openstack.org/#/c/558930/. This needs to be cleaned up
> so that we only check for and install the virtualenv(s) once and we
> need to handle mixed python2 and python3 environments better (so that
> you can run a python2 swift and python3 everything else).
>
> The other major issue we've run into is that nova file injection
> (which is tested by tempest) seems to require either libguestfs or
> nbd. libguestfs bindings for python aren't available on pypi and
> instead we get them from system packaging. This means if we want
> libguestfs support we have to enable system site packages when using
> virtualenvs. The alternative is to use nbd which apparently isn't
> preferred by nova and doesn't work under current devstack anyways.
>
> Why is this a problem? Well the new pip10 behavior that breaks
> devstack is pip10's refusable to remove distutils installed
> packages. Distro packages by and large are distutils packaged which
> means if you mix system packages and pip installed packages there is a
> good chance something will break (and it does break for current
> devstack). I'm not sure that using a virtualenv with system site
> packages enabled will sufficiently protect us from this case (but we
> should test it further). Also it feels wrong to enable system packages
> in a virtualenv if the entire point is avoiding system python
> packages.
>
> I'm not sure what the best option is here but if we can show that
> system site packages with virtualenvs is viable with pip10 and people
> want to move forward with devstack using a global virtualenv we can
> work to clean up this change and make it mergeable.

Now that pip 10 is here and we've got things relatively stable, it's
probably time to revisit this.

I think we should continue to explore the route that Clark has opened
up.  This isn't an emergency because all of the current devstack/pip10
conflicts have been resolved, however, there's no guarantee that
we won't add a new package with a conflict (which may be even more
difficult to resolve) or even that a future pip won't take an even
harder line.

I believe that installing all in one virtualenv has the advantage of
behaving more like what is expected of a project in the current python
ecosystem, while still retaining the co-installability testing that we
get with devstack.

What I'm a bit fuzzy on is how this impacts devstack plugins or related
applications.  However, it seems to me that we ought to be able to
essentially define the global venv as part of the API and then plugins
can participate in it.  Perhaps that won't be able to be automatic?
Maybe we'll need to set this up and then all devstack plugins will need
to change in order to use it?  If so, hopefully we'll be able to export
the functions needed to make that easy.

-Jim