Virtualenv (and Tox) broken when run under python<3.6
A recent release of importlib-resources (1.1.0) no longer works on python2.7 or python3.5. The issue is they import typing's ContextManager which didn't exist until python3.6 [0]. This means that python2 jobs and python3.5 jobs are currently unhappy if they need virtualenv. Unfortunately, many of our jobs use tox which uses virtualenv. One workaround being investigated [1] is to install importlib-resources==1.0.2 which does not try to use typing's ContextManager. If this is confirmed to work we will want to consider adding this change to the base job so that all jobs don't have to fix it separately. Note the version of python here is the one used to run virtualenv not the version of python being installed into the virtualenv. This means python3.6 running virtualenv to create a python2 virtualenv should be fine. But python3.5 running virtualenv to create a python3.6 env would not be fine. [0] https://gitlab.com/python-devs/importlib_resources/issues/83 [1] https://review.opendev.org/#/c/710729/ Clark
On Mon, Mar 2, 2020, at 9:12 AM, Clark Boylan wrote:
A recent release of importlib-resources (1.1.0) no longer works on python2.7 or python3.5. The issue is they import typing's ContextManager which didn't exist until python3.6 [0]. This means that python2 jobs and python3.5 jobs are currently unhappy if they need virtualenv. Unfortunately, many of our jobs use tox which uses virtualenv.
I noticed after sending the first email that it wasn't clear why tox and virtualenv are effected by an importlib-resources release. Virtualenv depends on importlib-resources [2] and tox uses virtualenv by default to create venvs. [2] https://github.com/pypa/virtualenv/blob/20.0.7/setup.cfg#L48
On Mon, Mar 2, 2020, at 9:12 AM, Clark Boylan wrote:
A recent release of importlib-resources (1.1.0) no longer works on python2.7 or python3.5. The issue is they import typing's ContextManager which didn't exist until python3.6 [0]. This means that python2 jobs and python3.5 jobs are currently unhappy if they need virtualenv. Unfortunately, many of our jobs use tox which uses virtualenv.
One workaround being investigated [1] is to install importlib-resources==1.0.2 which does not try to use typing's ContextManager. If this is confirmed to work we will want to consider adding this change to the base job so that all jobs don't have to fix it separately.
We've landed a version of this workaround, https://review.opendev.org/710851, to the base job in opendev/base-jobs. By default this is the base job that all zuul jobs inherit from. This appears to fix use of virtualenv and tox within jobs that use the globally installed versions of these tools. If you run a nested version of the tools (DIB chroot, containers, etc) you'll need to address this issue within that separate context.
Note the version of python here is the one used to run virtualenv not the version of python being installed into the virtualenv. This means python3.6 running virtualenv to create a python2 virtualenv should be fine. But python3.5 running virtualenv to create a python3.6 env would not be fine.
[0] https://gitlab.com/python-devs/importlib_resources/issues/83 [1] https://review.opendev.org/#/c/710729/
Clark
participants (1)
-
Clark Boylan