[all][dev] Beware how fun the new pip can be

Radosław Piliszek radoslaw.piliszek at gmail.com
Wed Jan 13 20:10:27 UTC 2021

A quick addendum after discussion with Clark (to make it easier to
digest for everyone).

The actors from PyPI:
msgpack - the latest is 1.0.2 and that is what is in the
upper-constraints that are being used
fluent-logger - the latest is 0.9.6 and this is being installed *unconstrained*
fluent-logger sets one dep: msgpack<1.0.0 since fluent-logger 0.9.5

The result:

1) old pip
and a WARNING that fluent-logger 0.9.6 wants msgpack<1.0.0

2) new pip
and no WARNINGs, no ERRORs, no anything, just happy silent "I got you
your package, so what if it is not the latest, I am the smart one
i.e. controlling *dependencies* controls *dependants*

And don't get me wrong, pip did what it advertised - took a list of
constraints and found a solution.

The outtake is simple: beware! :-)


On Wed, Jan 13, 2021 at 8:36 PM Radosław Piliszek
<radoslaw.piliszek at gmail.com> wrote:
> Hiya, Folks!
> Sharing what I have just learnt about the new pip's solver.
> pip install PROJECT no longer guarantees to install the latest version
> of PROJECT (or, well, giving you the ERROR that it cannot do it
> because something something :-) ).
> In fact, it will install the latest version *matching other
> constraints* and do it *silently*.
> Like it was recently only with Python version (i.e. py3-only would not
> get installed on py2 - that is cool) but now it moved into any-package
> territory.
> As an example, I can give you [1] where we are experimenting with
> getting some extracurricular package into our containers, notably
> fluent-logger.
> The only dep of fluent-logger is msgpack but the latest msgpack (as in
> upper constraints: 1.0.2, or any 1.x for that matter) is not
> compatible. However, the pin was introduced in fluent-logger in its
> 0.9.5 release (0.9.6 is the latest). Guess what pip does? Here is what
> it does:
> INFO:kolla.common.utils.openstack-base:Collecting fluent-logger
> INFO:kolla.common.utils.openstack-base:  Downloading
> http://mirror-int.dfw.rax.opendev.org:8080/pypifiles/packages/1a/f5/e6c30ec7a81e9c32c652c684004334187db4cc09eccf78ae7b69e62c7b10/fluent_logger-0.9.6-py2.py3-none-any.whl
> (12 kB)
> INFO:kolla.common.utils.openstack-base:  Downloading
> http://mirror-int.dfw.rax.opendev.org:8080/pypifiles/packages/d5/cb/19d838561ec210321aea24c496ec61930d6fdbb2f98d3f06cebab33c1331/fluent_logger-0.9.5-py2.py3-none-any.whl
> (12 kB)
> INFO:kolla.common.utils.openstack-base:  Downloading
> http://mirror-int.dfw.rax.opendev.org:8080/pypifiles/packages/d1/d4/f9b3493f974cdac831bf707c3d9fec93b1a0ebd986eae4db4f101dd72378/fluent_logger-0.9.4-py2.py3-none-any.whl
> (12 kB)
> And that's it. Pip is happy, you got your "latest" version.
> In previous pip one would get the latest version AND a warning. Now
> just pip's view on what the "latest" version is.
> I am glad we have upper-constraints which save the day here (forcing
> the ERROR) but beware of this "in the wild".
> [1] https://review.opendev.org/c/openstack/kolla/+/759855
> -yoctozepto

More information about the openstack-discuss mailing list