Open Stack

On 2020-12-14 09:54:53 +0000 (+0000), Lee Yarwood wrote:
> On 13-12-20 16:33:39, Jeremy Stanley wrote:
[...]
> > Tox doesn't reuse one virtualenv for multiple testenv
> > definitions, it creates a separate one for each, so for
> > example...
> 
> That isn't technically true within Nova, multiple tox envs use the
> {toxworkdir}/shared envdir for the virtualenv.
> 
> mypy, pep8, fast8, genconfig, genpolicy, cover, debug and bandit.
[...]

Neat, I suppose that's not a terrible workaround for some of this,
though I wonder if we'll see it cause problems over time with the
new dep solver in pip if some of those tools grow any conflicting
transitive dependencies.

> > I can't even run the pep8 testenv locally because to do that I
> > apparently need a Python package named zVMCloudConnector which wants
> > root access to create files like
> > /lib/systemd/system/sdkserver.service and
> > /etc/sudoers.d/sudoers-zvmsdk and /var/lib/zvmsdk/* and
> > /etc/zvmsdk/* in my system. WHAT?!? Do nova's developers actually
> > ever run any of this themselves?
> 
> ...
> 
> Which version of that package is the pep8 env pulling in for you?
> 
> I don't see any such issues with zVMCloudConnector==1.4.1 locally on
> Fedora 33, tox 3.19.0, pip 20.2.2 etc.
> 
> Would you mind writing up a launchpad bug for this?
[...]

I think I've worked out why you're not seeing it. My tox is
installed with the tox-venv plugin so that it will use the venv
module instead of virtualenv, and that doesn't seed a copy of the
wheel library into the testenv by default. Apparently if you try to
install zVMCloudConnector via `setup.py install` instead of making a
wheel (which is what happens by default if the wheel library is
absent), this is the result. For the curious, a simpler reproducer
is:

    rm -rf ~/.cache/pip  # in case you have a wheel cached for it
    python3 -m venv foo  # simple venv without the wheel module
    foo/bin/pip install zVMCloudConnector

Install wheel into the venv first and then zVMCloudConnector
installs cleanly, and indeed if I test with just plain tox (no
tox-venv plugin installed) it ends up getting a wheel for
zVMCloudConnector so doesn't hit the root-only build steps from its
sdist. A bit of research indicates tox-venv was deprecated earlier
this year once virtualenv gained the ability to delegate creation to
the venv module itself, and even if you set VIRTUALENV_CREATOR=venv
in the setenv list in tox.ini or passenv it from the calling
environment you're not going to run into this because venvs as built
from virtualenv get wheel seeded in them by default.

Now that I've gotten to the bottom of this, given it's a bit of a
corner case, I'm on the fence about filing a bug about it against
python-zvm-sdk in LP and likely won't unless folks actually think
it'll be useful to relate.

> Gibi, should we track all of this in a few launchpad bugs for Nova?

To be clear, I wasn't trying to single out nova, it was simply a
convenient example of where the idea of using a single
test-requirements.txt file may have tipped over from convenience
into inconvenience. (And then the zVMCloudConnector tangent of
course.)
-- 
Jeremy Stanley
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20201214/30e864f6/attachment.sig>