Open Stack

Hello,

This is to notify the community of the planned upcoming removal of the
"pip-and-virtualenv" element from our infra image builds.

This is part of the "cleanup test node python" spec documented at [1].

tl;dr
-----

- pip and virtualenv tools will no longer be pre-installed on the base
  images we use for testing, but will be installed by jobs themselves.
- most current jobs will depend on common Zuul base jobs that have
  been updated to install these tools already *i.e. in most cases
  there should be nothing to do*
- in general, jobs should ensure they depend on roles that install
  these tools if they require them
- if you do find your job failing due to the lack of either of these
  tools, use the ensure-pip or ensure-virtualenv roles provided in
  zuul-jobs
- we have "-plain" node types that implement this now.  If you think
  there might be a problem, switch the job to run run against one of
  these node types described below for testing.

History
-------

The "pip-and-virtualenv" element is part of diskimage-builder [2] and
installs the latest version of pip and virtualenv, and related tools
like setuptools, into the system environment for the daily builds.
This has a long history of working around various issues, but has
become increasingly problematic to maintain.

One of the more noticable effects is that to prevent the distribution
packages overwriting the upstream installs, we put various pip,
setuptools and virtualenv packages on "hold" in CI images.  This
prevents tests resinstalling packaged tools which can create, in
short, a big mess.  Both destroying the extant environment and having
the tools on hold have been problems for various projects at various
times.

Another other problem is what happens when you call "pip" or
"virtualenv" has diverged.  It used to be that "pip" would install the
package under Python2, while pip3 would install under python3.
However, modern distributions now have "pip" installing under Python
3.  To keep having "pip" install Python 2 packages on these platforms
is not just wrong, but drags in Python 2 dependences to our images
that aren't required.

The addition of Python 3's venv and the split with virtualenv makes
things even more confusing again.

Future
------

As famously said "the only winning move is not to play".  [3]

By dropping this element, images will not have non-packaged
pip/virtualenv/setuptools/etc. pre-installed.  No packages will be on
hold.

The "ensure-pip" role [4] will ensure that dependencies for "pip:"
commands will work.  Because of the way base jobs are setup, it is
most likely that this role has already run.  If you wish to use a
virtual environment to install a tool, I would suggest using the
"ensure_pip_virtualenv_cmd" this role exports.  This will default to
"python3 -m venv".  An example is [5].

In a Python 3 word, you probably do not *need* virtualenv; "python3 -m
venv" is available and works after the "ensure-pip" role is run.
However, if you require some of the features that virtualenv provides
that venv does not (explained at [6]) there is a role
"ensure-virtualenv" [7].  For example, we do this on the devstack
branches because it is common to use "virtualenv" there due to the
long history [8].

If you need specific versions of pip or virtualenv, etc. beyond the
system-packaged versions installed with the above, you should have
your job configure these.  There is absolutely no problem with jobs
installing differing versions of pip/virtuatlenv/etc in any way they
want -- we just don't want the base images to have any of that by
default.  Of course, you should consider if you're building/testing
something that is actually useful outside the gate, but that is a
global concern.

Testing
-------

We have built parallel nodes with the suffix "-plain" where you can
test any jobs in the new environment.  For example [9] speculatively
tests devstack.  The node types available are

 centos-7-plain
 centos-8-plain
 ubuntu-xenial-plain
 ubuntu-bionic-plain

The newer focal images do not have pip pre-installed, neither do the
faster moving Fedora images, any SUSE images, or any ARM64 images.

Rollout
-------

We would like to make the switch soon, to shake out any issues early
in the cycle.  This would mean on or about the 8th June.

Thanks,

-i

[1] https://docs.opendev.org/opendev/infra-specs/latest/specs/cleanup-test-node-python.html
[2] https://opendev.org/openstack/diskimage-builder/src/branch/master/diskimage_builder/elements/pip-and-virtualenv
[3] https://en.wikipedia.org/wiki/WarGames
[4] https://zuul-ci.org/docs/zuul-jobs/python-roles.html#role-ensure-pip
[5] https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/bindep/tasks/install.yaml#L9
[6] https://virtualenv.pypa.io/en/latest/
[7] https://zuul-ci.org/docs/zuul-jobs/python-roles.html#role-ensure-virtualenv
[8] https://opendev.org/openstack/devstack/commit/23cfb9e6ebc63a4da4577c0ef9e3450b9c946fa7
[9] https://review.opendev.org/#/c/712211/11/.zuul.yaml