On Tue, 8 Dec 2020 at 18:24, Jeremy Stanley <fungi@yuggoth.org> wrote:
On 2020-12-08 13:36:09 +0100 (+0100), Spyros Trigazis wrote:
openstack-tox-lower-constraints fails for bashate and coverage. (Maybe more, I bumped bashate and it failed for coverage. I don;t want to waste more resources on our CI) eg https://review.opendev.org/c/openstack/magnum/+/765881 https://review.opendev.org/c/openstack/magnum/+/765979
Do we miss something?
Pip 20.3.0, released 8 days ago, turned on a new and much more thorough dependency resolver. Earlier versions of pip did not try particularly hard to make sure the dependencies claimed by packages were all satisfied. Virtualenv 20.2.2 released yesterday and increased the version of pip it's vendoring to a version which uses the new solver as well. These changes mean that latent version conflicts are now being correctly identified as bugs, and these jobs will do a far better job of actually confirming the declared versions of dependencies are able to be tested.
One thing which looks really weird and completely contradictory to me is that your lower-constraints job on change 765881 is applying both upper and lower constraints lists to the pip install command. Maybe the lower constraints list is expected to override the earlier upper constraints, but is that really going to represent a compatible set? That aside, trying to reproduce locally I run into yet a third error:
Could not find a version that satisfies the requirement warlock!=1.3.0,<2,>=1.0.1 (from python-glanceclient)
And indeed, python-glanceclient insists warlock 1.3.0 should be skipped, while magnum's lower-constraints.txt says you must install warlock==1.3.0 so that's a clear contradiction as well.
My recommendation is to work on reproducing this locally first and play a bit of whack-a-mole with the entries in your lower-constraints.txt to find versions of things which will actually be coinstallable with current versions of pip. You don't need to run the full tox testenv, just try installing your constrainted deps into a venv with upgraded pip like so:
python3.8 -m venv foo foo/bin/pip install -U pip foo/bin/pip install -c lower-constraints.txt \ -r test-requirements.txt -r requirements.txt
You'll also likely want to delete and recreate the venv each time you try, since pip will now also try to take the requirements of already installed packages into account, and that might further change the behavior you see. Hope that helps! -- Jeremy Stanley
Hi Jeremy, Sorry for hijacking this thread, but I am dealing with the same sort of issues in blazar. The lower-constraints job is failing [1], which has broken our gate. However, I cannot reproduce the issue locally: `tox -e lower-constraints` works fine, and so do your recommended commands with the foo venv. I've tried on multiple operating systems, including Ubuntu 20.04 (with Python 3.8 and pip 20.3.3). So I have been playing whack-a-mole [2] from Zuul logs based on whichever package fails to install, but it's a very slow process. Do you know what I could be missing for reproducing this locally? Thanks, Pierre [1] https://zuul.opendev.org/t/openstack/build/34897587cf954016b8027670117213f6 [2] https://review.opendev.org/c/openstack/blazar/+/767593/