---- On Fri, 11 Dec 2020 09:58:19 -0600 Sean McGinnis <sean.mcginnis@gmx.com> wrote ----
Yes, it bears repeating that anywhere the new dep solver is breaking represents a situation where we were previously testing/building things with a different version of some package than we meant to. This is exposing latent bugs in our declared dependencies within those branches. If we decide to use "older" pip, that's basically admitting we don't care because it's easier to ignore those problems than actually fix them (which, yes, might turn out to be effectively impossible). I'm not trying to be harsh, it's certainly a valid approach, but let's be clear that this is the compromise we're making in that case.
+1
My proposal: actually come to terms with the reality that lower-constraints jobs are a fundamentally broken concept, unless someone does the hard work to implement an inverse version sort in pip itself. If pretty much all the struggle is with those jobs, then dropping them can't hurt because they failed at testing exactly the thing they were created for in the first place.
As someone that has spent some time working on l-c jobs/issues, I kind of have to agree with this.
For historical reference, here's the initial proposal for performing lower constraint testing:
http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
I wasn't part of most of the requirements team discussions around the need for this, but my understanding was it was to provide a set range of package versions that are expected to work. So anyone packaging OpenStack projects downstream would have an easy way to filter out options to figure out what versions they can include that will minimize conflicts between all the various packages included in a distro.
I'm not a downstream packager, so I don't have any direct experience to go on here, but my assumption is that the value of providing this is pretty low. I don't think the time the community has put in to trying to maintain (or not maintain) their lower-constraints.txt files and making sure the jobs are configured properly to apply those constraints has been worth the effort for the value anyone downstream might get out of them.
My vote would be to get rid of these jobs. Distros will need to perform testing of the versions they ultimately package together anyway, so I don't think it is worth the community's time to repeatedly struggle with keeping these things updated.
I do think one useful bit can be when we're tracking our own direct dependencies. One thing that comes to mind from the recent past is we've had cases where something new has been added to something like oslo.config. Lower constraints updates were a good way to make it explicit that we needed at least the newer version of that lib so that we could safely assume the expected functionality would be present. There is some value to that. So if we keep lower-constraints, maybe we just limit it to those specific instances where we have things like that and not try to constrain the entire world.
I agree. One of the big chunks of work and time we spent on this is during moving the testing from Ubuntu Bionic to Focal[1] where we had to fix many lower-constraints in all the repos (nearly ~400) in OpenStack. For knowing the compatible version of deps, we have the information in the requirements.txt file, where we can get to know what is working version of all deps are. What is the min working version is not a hard thing to know (basically when env break for any incompatible one). Maintaining it up to date is not so worth compare to the effort it is taking. I will also suggest to remove this. [1] https://storyboard.openstack.org/#!/story/2007865 -gmann
Sean