On Tue, May 12, 2020, at 7:44 AM, Sorin Sbarnea wrote:
I recently discovered that this problem goes even deeper, please read all comments on https://github.com/tox-dev/tox/issues/1565
Due to this it seems to be impossible to define zuul jobs that use a specific python version regardless the environment name. If you want to force "linters" job to use python3.8 only under CI/CD, is impossible.
Using basepython=pythonX.Y combined with ignore_basepython_conflict=False seems the only way to enforce version use, and it comes at the cost of not being flexible for developers (as they may not have the exact version that we want to use on CI jobs).
While for unittest jobs we do use pyXY in the environment name, we do not have the same for "linters", "docs". Alternative to add `{pyXY}-linters` seems to be the only option to trick it.
I don't think that is a trick, this is an intentional feature of tox to solve the problem you have. If you want a specific version of python to be used tox aims to help you do that via the pyXY* environments. https://tox.readthedocs.io/en/latest/config.html#tox-environments clearly describes this behavior.
Another (dangerous) approach would be to assure that the only python version available on each nodeset used by tox job is the one we want to use for testing.
I engaged with Bernat Gabor (tox maintainer) on Gitter/tox-dev about this issue and apparently there is no solution. Tox v4 has some changes planned but is far away.
* --discover cannot be used to enforce python detection * tox own interpreter version cannot be used to enforce version being picked (at least this is what the maintainer told me)
Sadly, after spending a good number of hours on that I am more confused that I was when I started.
With Zuul nodepool images that can change over night on multiple zuul servers (at least 4 i use), it seems that tox-* jobs are joing to be a permanent source of surprises, where we can discover that what they say they test is not really what they did.
In the case of linting, without a specific python version set, they are doing what you asked: run the linters without a specific version of python and use what is available. The problem I described earlier was asking for a specific python version and not getting that version. Running tox -e linters does not ask for a specific version. The workaround you described above is the actual tox solution to this problem. You should run py38-linters if that is your intent.
Anyone missing a `make foo` command? ;)
Cheers Sorin