On Apr 18, 2020, at 1:29 PM, Jeremy Stanley <fungi@yuggoth.org> wrote:
On 2020-04-18 18:11:58 +0200 (+0200), Andreas Jaeger wrote:
More fun:
change https://review.opendev.org/720877 removes:
install_command = pip install {opts} {packages}
And suddenly lower-constraints fails - and the failure looks correct, so removing it is fine but will cause a surprise ;(
After analyzing this over IRC, it looks like openstack/networking-hyperv probably has an incoherent lower-constraints.txt file because it asks for (one of many examples) neutron-lib==1.28.0 and oslo.concurrency==3.25.0 but neutron-lib 1.28.0 declares it requires oslo.concurrency>=3.26.0 so this won't work.
The lower constraints jobs are built on the premise that they're working with a satisfiable set of constraints. Any time you alter a lower bound for any dependency or add a new dependency, you'll need to *fully* rebuild your lower-constraints.txt because of the possible knock-on effects such a change can have on the transitive dependencies tracked there.
https://review.opendev.org/675572 replaced neutron-lib>=1.18.0 with neutron-lib>=1.28.0 in the requirements.txt, but only edited the corresponding entry in lower-constraints.txt instead of taking all of neutron-lib's dependencies (and their dependencies, and so on) into account. The result is a lower constraints job which isn't testing what they think it's testing.
The good news is that's all fixable. The bad news is that this problem is probably widespread, because the places I expected to talk about how to properly maintain your lower-constraints.txt file provide minimal and potentially misleading advice:
https://docs.openstack.org/project-team-guide/dependency-management.html#add...
https://docs.openstack.org/project-team-guide/dependency-management.html#upd...
Neither of those suggest rebuilding your lower-constraints.txt when you alter a lower bound or add a new requirement. Further, I thought I remembered there being a utility in the openstack/requirements repository for correctly generating a lower-constraints.txt file, but this does not actually exist that I can find.
IIRC, the scripting I did way back when [1] was pretty naive. It looked at requirements.txt, converted >= to == and assumed either that any second order dependencies installed as a result of that set of constraints were the lowest likely to work or that teams would take care of editing the list. There did used to be an update-requirements tool in the requirements repo, but it was removed as part of that work and I’m not sure it ever dealt with lower bounds. [1] http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
The more I think back, the more I'm starting to remember that I suggested the only thorough way (short of reimplementing pip's version selection logic ourselves) to generate a lower constraints file from the lower bounds in a requirements set would be by using an altered copy of pip which inverted its version comparisons so that it selected the lowest available versions of packages which would satisfy every version range rather than the highest. I don't think anybody has made that, and so I expect instead we've collectively hand-waved with an assumption that people would be able to figure out a working set of lower constraints instead (which is in my opinion rather unrealistic given the complexity of the transitive dependency trees of even some of our medium-sized projects).
The upshot of this is that correctly calculating a lower constraint set for any project is likely to involve a lot of trial and error, probably far more than most projects are going to want to bear. If we had a tool we could run to generate a coherent lower-constraints.txt for a project, then this might be different, but I suspect that too is going to be more hassle than anyone wants to invest their time in creating.
Yes, it seems that very few distributions actually want the *lowest* values, either. What they seem to want is something that matches what they are already packaging, which is likely to be less than the version in the upper constraints list.
-- Jeremy Stanley