Open Stack

On Wed, Apr 19, 2017, at 05:54 AM, Julien Danjou wrote:
> Hoy,
> 
> So Gnocchi gate is all broken (agaiiiin) because it depends on "pbr" and
> some new release of oslo.* depends on pbr!=2.1.0.
> 
> Neither Gnocchi nor Oslo cares about whatever bug there is in pbr 2.1.0
> that got in banished by requirements Gods. It does not prevent it to be
> used e.g. to install the software or get version information. But it
> does break anything that is not in OpenStack because well, pip installs
> the latest pbr (2.1.0) and then oslo.* is unhappy about it.

It actually breaks everything, including OpenStack. Shade and others are
affected by this as well. The specific problem here is that PBR is a
setup_requires which means it gets installed by easy_install before
anything else. This means that the requirements restrictions are not
applied to it (neither are the constraints). So you get latest PBR from
easy_install then later when something checks the requirements
(pkg_resources console script entrypoints?) they break because latest
PBR isn't allowed.

We need to stop pinning PBR and more generally stop pinning any
setup_requires (there are a few more now since setuptools itself is
starting to use that to list its deps rather than bundling them).

> So I understand the culprit is probably pip installation scheme, and we
> can blame him until we fix it. I'm also trying to push pbr 2.2.0 to
> avoid the entire issue.

Yes, a new release of PBR undoing the "pin" is the current sane step
forward for fixing this particular issue. Monty also suggested that we
gate global requirements changes on requiring changes not pin any
setup_requires.

> But for the future, could we stop updating the requirements in oslo libs
> for no good reason? just because some random OpenStack project hit a bug
> somewhere?
> 
> For example, I've removed requirements update on tooz¹ for more than a
> year now, which did not break *anything* in the meantime, proving that
> this process is giving more problem than solutions. Oslo libs doing that
> automatic update introduce more pain for all consumers than anything (at
> least not in OpenStack).

You are likely largely shielded by the constraints list here which is
derivative of the global requirements list. Basically by using
constraints you get distilled global requirements and even without being
part of the requirements updates you'd be shielded from breakages when
installed via something like devstack or other deployment method using
constraints.

> So if we care about Oslo users outside OpenStack, I beg us to stop this
> crazyness. If we don't, we'll just spend time getting rid of Oslo over
> the long term…

I think we know from experience that just stopping (eg reverting to the
situation we had before requirements and constraints) would lead to
sadness. Installations would frequently be impossible due to some
unresolvable error in dependency resolution. Do you have some
alternative in mind? Perhaps we loosen the in project requirements and
explicitly state that constraints are known to work due to testing and
users should use constraints? That would give users control to manage
their own constraints list too if they wish. Maybe we do this in
libraries while continuing to be more specific in applications?

> 
> My 2c,
> 
> Cheers,
> 
> ¹ Unless some API changed in a dep and we needed to raise the dep,
> obviously.
> 
> -- 
> Julien Danjou
> # Free Software hacker
> # https://julien.danjou.info

I don't have all the answers, but am fairly certain the situation we
have today is better than the one from several years ago. It is just not
perfect. I think we are better served by refining the current setup or
replacing it with something better but not by reverting.

Clark