Open Stack

On Tue, Feb 17, 2015 at 4:19 AM, Sean Dague <sean at dague.net> wrote:

> On 02/16/2015 08:50 PM, Ian Cordasco wrote:
> > On 2/16/15, 16:08, "Sean Dague" <sean at dague.net> wrote:
> >
> >> On 02/16/2015 02:08 PM, Doug Hellmann wrote:
> >>>
> >>>
> >>> On Mon, Feb 16, 2015, at 01:01 PM, Ian Cordasco wrote:
> >>>> Hey everyone,
> >>>>
> >>>> The os-ansible-deployment team was working on updates to add support
> >>>> for
> >>>> the latest version of juno and noticed some interesting version
> >>>> specifiers
> >>>> introduced into global-requirements.txt in January. It introduced some
> >>>> version specifiers that seem a bit impossible like the one for
> requests
> >>>> [1]. There are others that equate presently to pinning the versions of
> >>>> the
> >>>> packages [2, 3, 4].
> >>>>
> >>>> I understand fully and support the commit because of how it improves
> >>>> pretty much everyone’s quality of life (no fires to put out in the
> >>>> middle
> >>>> of the night on the weekend). I’m also aware that a lot of the
> >>>> downstream
> >>>> redistributors tend to work from global-requirements.txt when
> >>>> determining
> >>>> what to package/support.
> >>>>
> >>>> It seems to me like there’s room to clean up some of these
> requirements
> >>>> to
> >>>> make them far more explicit and less misleading to the human eye (even
> >>>> though tooling like pip can easily parse/understand these).
> >>>
> >>> I think that's the idea. These requirements were generated
> >>> automatically, and fixed issues that were holding back several
> projects.
> >>> Now we can apply updates to them by hand, to either move the lower
> >>> bounds down (as in the case Ihar pointed out with stevedore) or clean
> up
> >>> the range definitions. We should not raise the limits of any Oslo
> >>> libraries, and we should consider raising the limits of third-party
> >>> libraries very carefully.
> >>>
> >>> We should make those changes on one library at a time, so we can see
> >>> what effect each change has on the other requirements.
> >>>
> >>>>
> >>>> I also understand that stable-maint may want to occasionally bump the
> >>>> caps
> >>>> to see if newer versions will not break everything, so what is the
> >>>> right
> >>>> way forward? What is the best way to both maintain a stable branch
> with
> >>>> known working dependencies while helping out those who do so much work
> >>>> for
> >>>> us (downstream and stable-maint) and not permanently pinning to
> certain
> >>>> working versions?
> >>>
> >>> Managing the upper bounds is still under discussion. Sean pointed out
> >>> that we might want hard caps so that updates to stable branch were
> >>> explicit. I can see either side of that argument and am still on the
> >>> fence about the best approach.
> >>
> >> History has shown that it's too much work keeping testing functioning
> >> for stable branches if we leave dependencies uncapped. If particular
> >> people are interested in bumping versions when releases happen, it's
> >> easy enough to do with a requirements proposed update. It will even run
> >> tests that in most cases will prove that it works.
> >>
> >> It might even be possible for someone to build some automation that did
> >> that as stuff from pypi released so we could have the best of both
> >> worlds. But I think capping is definitely something we want as a
> >> project, and it reflects the way that most deployments will consume this
> >> code.
> >>
> >>      -Sean
> >>
> >> --
> >> Sean Dague
> >> http://dague.net
> >
> > Right. No one is arguing the very clear benefits of all of this.
> >
> > I’m just wondering if for the example version identifiers that I gave in
> > my original message (and others that are very similar) if we want to make
> > the strings much simpler for people who tend to work from them (i.e.,
> > downstream re-distributors whose jobs are already difficult enough). I’ve
> > offered to help at least one of them in the past who maintains all of
> > their distro’s packages themselves, but they refused so I’d like to help
> > them anyway possible. Especially if any of them chime in as this being
> > something that would be helpful.
>
> Ok, your links got kind of scrambled. Can you next time please inline
> the key relevant content in the email, because I think we all missed the
> original message intent as the key content was only in footnotes.
>
> From my point of view, normalization patches would be fine.
>
> requests>=1.2.1,!=2.4.0,<=2.2.1
>
> Is actually an odd one, because that's still there because we're using
> Trusty level requests in the tests, and my ability to have devstack not
> install that has thus far failed.
>
> Things like:
>
> osprofiler>=0.3.0,<=0.3.0 # Apache-2.0
>
> Can clearly be normalized to osprofiler==0.3.0 if you want to propose
> the patch manually.
>

global-requirements for stable branches serves two uses:

1. Specify the set of dependencies that we would like to test against
2.  A tool for downstream packagers to use when determining what to
package/support.

For #1, Ideally we would like a set of all dependencies, including
transitive, with explicit versions (very similar to the output of
pip-freeze). But for #2 the standard requirement file with a range is
preferred. Putting an upper bound on each dependency, instead of using a
'==' was a compromise between the two use cases.

Going forward I propose we have a requirements.in and a requirements.txt
file. The requirements.in file would contain the range of dependencies, and
requirements.txt would contain the pinned set, and eventually the pinned
set including transitive dependencies.

Thoughts?


>
>         -Sean
>
> --
> Sean Dague
> http://dague.net
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150217/06d92cad/attachment.html>