[all] Lower-constraints in some projects broken - update your repos

Radosław Piliszek radoslaw.piliszek at gmail.com
Sat Apr 18 08:41:16 UTC 2020

On Fri, Apr 17, 2020 at 9:06 PM Sean McGinnis <sean.mcginnis at gmx.com> wrote:
> This had to be explained to me a few times before I got it, so don't
> feel bad. ;)

No problem, I don't. :-) Cannot feel bad when docs say what they say.

> The lower-constraints file really does need to include everything, not
> just the project's direct dependencies. If it only contains what is
> listed in requirements.txt, then there is the possibility that one of
> the indirect dependencies that does not strictly enforce its own
> required version will end up pulling in a newer version than what is
> expected.
> The goal of lower constraints testing is to have a static set of all
> requirements that are known and validated to work. If we only restrict
> the 1st level dependencies, then without changing anything, that known
> set that was passing tests could suddenly stop working.
> We actually had a good discussion of this on IRC today, so it might be
> helpful to read that for a little more context:
> http://eavesdrop.openstack.org/irclogs/%23openstack-sdks/%23openstack-sdks.2020-04-17.log.html#t2020-04-17T15:33:38

I read it all and agree with Dmitry that this would make most sense if
we had to pin only direct deps (which is what I did exactly).
However, I see the point that pip has shortcomings in dependency
resolution and may end up with broken dependency chain.

> I think ultimately the goal when this was added was to be able to
> communicate downstream an acceptable range of compatible packages that
> could be installed together and expect to work.

Let's ask packagers if they ever used this info. Maybe we are doing
something just for doing it (and not really *doing* it actually). ;-)

> We could probably be more aggressive about raising those minimums to
> keep picking up newer things. But unless we can get a static(ish)
> snapshot of the whole dependency graph like this, even that would like
> break often. Or not accurately test what we think we are testing.

These are pain points for sure.
On the one hand, it looks like lower constraints tries to be like
npm/yarn lockfile in nodejs world [1] [2].
On the other, that it is to ensure our lower constraints are modern
enough to handle our code.
But it seemingly falls short doing either, because the first is not
enforced, and the second is additionally limited in that we only run
unit testing (as far as I could see) where real deps testing will
actually happen in functional testing (as in unit testing a ton of
functionality is mocked anyway, so we may often end up testing
importability at most).

All in all, I think we discovered shortcomings in our methodology,
tooling and docs, and it's a good topic for meeting agenda (reqs team?
QA? TC?).

[1] https://docs.npmjs.com/configuring-npm/package-locks.html
[2] https://classic.yarnpkg.com/en/docs/yarn-lock/


More information about the openstack-discuss mailing list