On 2020-12-14 09:54:53 +0000 (+0000), Lee Yarwood wrote:
On 13-12-20 16:33:39, Jeremy Stanley wrote: [...]
Tox doesn't reuse one virtualenv for multiple testenv definitions, it creates a separate one for each, so for example...
That isn't technically true within Nova, multiple tox envs use the {toxworkdir}/shared envdir for the virtualenv.
mypy, pep8, fast8, genconfig, genpolicy, cover, debug and bandit. [...]
Neat, I suppose that's not a terrible workaround for some of this, though I wonder if we'll see it cause problems over time with the new dep solver in pip if some of those tools grow any conflicting transitive dependencies.
I can't even run the pep8 testenv locally because to do that I apparently need a Python package named zVMCloudConnector which wants root access to create files like /lib/systemd/system/sdkserver.service and /etc/sudoers.d/sudoers-zvmsdk and /var/lib/zvmsdk/* and /etc/zvmsdk/* in my system. WHAT?!? Do nova's developers actually ever run any of this themselves?
...
Which version of that package is the pep8 env pulling in for you?
I don't see any such issues with zVMCloudConnector==1.4.1 locally on Fedora 33, tox 3.19.0, pip 20.2.2 etc.
Would you mind writing up a launchpad bug for this? [...]
I think I've worked out why you're not seeing it. My tox is installed with the tox-venv plugin so that it will use the venv module instead of virtualenv, and that doesn't seed a copy of the wheel library into the testenv by default. Apparently if you try to install zVMCloudConnector via `setup.py install` instead of making a wheel (which is what happens by default if the wheel library is absent), this is the result. For the curious, a simpler reproducer is: rm -rf ~/.cache/pip # in case you have a wheel cached for it python3 -m venv foo # simple venv without the wheel module foo/bin/pip install zVMCloudConnector Install wheel into the venv first and then zVMCloudConnector installs cleanly, and indeed if I test with just plain tox (no tox-venv plugin installed) it ends up getting a wheel for zVMCloudConnector so doesn't hit the root-only build steps from its sdist. A bit of research indicates tox-venv was deprecated earlier this year once virtualenv gained the ability to delegate creation to the venv module itself, and even if you set VIRTUALENV_CREATOR=venv in the setenv list in tox.ini or passenv it from the calling environment you're not going to run into this because venvs as built from virtualenv get wheel seeded in them by default. Now that I've gotten to the bottom of this, given it's a bit of a corner case, I'm on the fence about filing a bug about it against python-zvm-sdk in LP and likely won't unless folks actually think it'll be useful to relate.
Gibi, should we track all of this in a few launchpad bugs for Nova?
To be clear, I wasn't trying to single out nova, it was simply a convenient example of where the idea of using a single test-requirements.txt file may have tipped over from convenience into inconvenience. (And then the zVMCloudConnector tangent of course.) -- Jeremy Stanley