Open Stack

On 30 July 2015 at 13:12, Clay Gerrard <clay.gerrard at gmail.com> wrote:
> I agree an error message is better than breaking for insane reasons.
>
> But... maybe as an aside... what about not breaking?
>
> How come the openstack ecosystem doesn't have wait for PEP 426 to be
> approved and for setuptools 17.1 to be widely deployed before it can
> require/depend on it?  Is there no failure/degraded case where we can take
> advantage of moving forward in the infrastructure where we apparently need
> it - but not necessarily force everyone else to upgrade if it's not directly
> solving something for them (e.g. people doing packaging of openstack
> projects, but don't personally necessarily currently maintain or distribute
> a setuptools package)?
>
> On the web this happens all the time "Sure maybe you can't get the newest
> HTML5 wiz-bang, but I can still render something on IE8, OTOH, IE7, wow -
> pls upgrade" vs. "How are you not even running setuptools 17.1 right now in
> your build environment - it was *literally* released *almost* two months
> ago!?  I... I can't even... it hurts to look at you."
>
> Just Curious.
>
> I only recently found out that PEP 426 was a thing, so I think it's pretty
> great to see people driving the python packaging ecosystem forward.  For
> those involved.  Kudos.

So, we [folk in OpenStack that are worried about our packaging] got
involved with upstream pip, setuptools, PEP-426 etc after some years
of trying the 'use whats ready, and workaround the rest by layering on
top' approach.

That approach led us to:
 - a nearly diametrically opposite definition of requirements files,
which confuses approximately everyone that reads the pip docs and not
the pbr ones - and vice versa.
 - a pattern of emergent bugs where we failed to consider a corner
case because we worked around the surface, which emerges as a  bad
behaviour. For instance, our non-compliant version numbers in pbr <
0.10.8 (IIRC the point # correctly). Or the way requirements-py3.txt
files caused use to create universal wheels with bad metadata that
couldn't install properly on other Python releases than the one they
were built on.
 - a chunk of overlapping effort, where setuptools and pip were
tackling the same stuff we were, because we're not that special a
snowflake: many folk experience the pains we feel from packaging
limits...  but not benefiting from what we'd learnt, nor from the
effort we put in.

So - for OpenStack as a whole, just consuming the current state and
working around was a buggy expensive strategy. And *even with that* we
still required the latest release of pip, setuptools and virtualenv
simply because we would run into a bug, get it fixed, and need to
consume it in devstack: devstack installs the latest of these four
(add pbr) crucial packaging tools unconditionally and has for a long
time.

So right now I, and a few others, are pushing on:
 - Aligning our designs with the ecosystem for less waste and
confusion all around.
 - Doing the work we had previously done as layered workarounds as
root level patches to the various tools

While still rolling out the new things, whatever they are, within
OpenStack before we consider them delivered.

HTH,
Rob

-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Converged Cloud