Open Stack

On Wed, Feb 27, 2013, Monty Taylor <mordred at inaugust.com> wrote:
> I totally understand the argument against stricter pinning, but I'd like
> to repeat my argument for it.
> 
> That is - while in theory it might seem nice to be able to "get upgrades
> for free" from upstreams, experience in this project has shown over and
> over that it's almost always a disaster. It _shouldn't_ be - but it is.
[snip]
> >     3) order in pip-requires matters :)
> > 
> >     The eventual fix for python-quantumclient was to bump cliff. There
> >     was nothing wrong with cliff, but cliff depends on pyparsing as
> >     well, and didn't specify strict version until 1.3.1. In reality this
> >     probably wasn't even a needed change for the pypi package, but it
> >     did let it get through our pip based pep8 scans. 
> > 
> >     The CI fix should really have been to move pyparsing up in the file
> >     to install before cliff, and 1.2.1 would then still work.

> We were considering the other day, when talking about the
> openstack/requirements repo, writing a script that would recursively
> expand all of the requirements of the requirements and place all of them
> in the master list so that had an explicit list of everything.

We would need better tooling to deal with this sanely or probably every
dependency will lag behind because of the pain in updating this list of
recursively expanded dependencies.

Even better if new versions of dependencies can be tested automatically,
so we can get bug fix releases into the list faster.

One upshot is we can see the entire list of dependencies. I was
surprised the first time I saw tox installing NumPy because of a
(technically optional) dependency websockify has on it.

Alternatively, we can accept that fact that breakage can and will occur
and optimize the process for recovering from that breakage faster. The
downside to this is possibly requiring tarball releases faster for
stable releases.

Ideally, this could be all solved by pypi software better documenting
their versioning scheme. We could then more intelligently specify
dependencies onto APIs and not versions. Unfortunately this requires
quite a community effort to make sure everyone follows this reliably.

JE