Open Stack

On Mon, Aug 31, 2020 at 12:27:33PM +1000, Ian Wienand wrote:
> Thus it is unfortunately not just s,/usr/local/bin,/usr/bin,g because
> the new setuptools will install all the libraries into site-packages;
> which the packaged python intpreter doesn't know to look for.

https://review.opendev.org/748937 was where I tried this before I
understood the above.

> Using SETUPTOOLS_USE_DISTUTILS=stdlib with such installs is one
> option; it feels like it just makes for more confusing bifurcation.
> We can't really set this in stack.sh as a global, because we wouldn't
> want this to apply to subshells that are installing in virtualenv's,
> for example.  It might be the best option.

https://review.opendev.org/748957 is this option; this should hook
into pip_install function.  How many plugins do "sudo pip install ..."
I don't know; they would all still be broken with this.  But as
mentioned, we don't want to set this globally to avoid setting it for
virtualenv installs.

> A more radical thought; perhaps we could install a non-packaged python
> interpreter for devstack runs.  Isolation from packaging cuts both
> ways; while we might work around packaging issues in CI, we're also
> working around packaging issues that then just hit you when you're in
> production.  The eternal question of "what are we testing".

On further consideration I don't think this is a great idea.  Lots of
things do #!/usr/bin/python3 which is always going to be the packaged
Python.  I imagine we'd have quite a mess of things not understanding
which python their libraries are installed for.

Another thing that failed was just using the system packaged pip;
https://review.opendev.org/748942.  In theory that would be OK, and
obviously patched correctly for the distro, but unfortunately the
bionic pip is so old it doesn't pull down manylinux2010 wheels and so
there's assorted build breakages from packages that now have to build.

https://review.opendev.org/748943/ is a pin to <50 in requirements.
devstack uses requirements to install setuptools in it's
tools/install_pip.sh so this does move the system back to a version
without this change.  Obviously this doesn't fix the underlying
problem, but helps the gate.

-i