Open Stack

On 2021-01-27 10:00:14 +0000 (+0000), Stephen Finucane wrote:
> On Thu, 2021-01-21 at 14:50 +0000, Jeremy Stanley wrote:
> > On 2021-01-21 09:30:19 +0000 (+0000), Stephen Finucane wrote:
> > [...]
> > > What we have doesn't work, and direct dependencies are the
> > > only things we can truly control. In the scenario you're
> > > suggesting, not only do we need to track dependencies, but we
> > > also need to track the dependencies of dependencies, and the
> > > dependencies of the dependencies of the dependencies, and the
> > > dependencies of the dependencies of the dependencies of the
> > > dependencies etc. etc. down the rabbit hole. For each of these
> > > indirect dependencies, of which there may be many, we need to
> > > figure out what the minimum version is for each of these
> > > indirect dependencies is manually, because as has been noted
> > > many times there is no standardized machinery in place in pip
> > > etc. to find (and test) the minimum dependency versions
> > > supported by a package. Put another way, if we depend on
> > > package foo, which depends on package bar, which depends on
> > > package baz, we can state our own informed minimum version for
> > > foo, but we will need to inspect foo to find a minimum version
> > > of bar that is suitable, and we will need to inspect baz to
> > > find a minimum version of baz that is suitable. An impossible
> > > ask.
> > [...]
> > 
> > Where this begins to fall apart, as I mentioned earlier, is that
> > the larger your transitive dependency set, the more likely it is
> > that a direct dependency is *also* an indirect dependency (maybe
> > many layers down). If a dependency of your dependency updates to
> > a version which insists on a newer version of some other direct
> > dependency of yours than what you've set in
> > lower-constraints.txt, then your jobs are going to break and
> > need lower bounds adjustments or additional indirect
> > dependencies added to the lower-constraints.txt to roll them
> > back to versions which worked with the others you've set. Unlike
> > upper-constraints.txt where it's assumed that a complete
> > transitive set of dependencies is covered, this will mean
> > additional churn in your stable branches over time.
> 
> Ah, I didn't grasp this particular point when you initially raised
> it. I've spent some time playing around with pip (there's a fun
> code base) to no avail, so this (the reduced dependency set) is
> still the best idea I've got, flaws and all. With that said, our
> 'upper-constraints.txt' should still capture all dependencies,
> including those indirect ones, no? As such, none of those should
> be increasing on stable branches, which means the set of
> transitive dependencies should remain fixed once the branch is
> cut?
> 
> > Or is the idea that we would only every do lower bounds checking
> > on the release under development, and then remove those jobs
> > when we branch?
> 
> This is also an option if it proves to be particularly painful. It
> does feel like this would indicate a failure of our
> upper-constraints to cap a dependency, even if its indirect, but I
> realize there are limits to what we can achieve here.

Well, that's perhaps an option, you're basically suggesting
combining the complete global upper-constraints.txt from
openstack/requirements with an incomplete local
lower-constraints.txt in each project. That should allow you to have
a complete transitive set, however how you would go about
integrating them together would still need to be determined.
-- 
Jeremy Stanley
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210127/f32caadb/attachment-0001.sig>