Open Stack

Thomas,

I normally side with the distro's take on making sure there is no duplication, but I think Thierry's point comes from two differences coming up that the traditional distro's don't tend to account for.

The traditional distro usually is about a single machine. being able to install all the things together. This often is beneficial but sometimes has major conflicts. OpenStack deployments often run into issues where you might want to run a newer X or newer Y package that is compatible. For example, I often want to run a liberty nova and a mitaka horizon on the same machine. Distro's currently make that impossible with their deployment tools.

There are a few ways around this... Some sites use multiple physical or virtual machines to separate the distro's components so that incompatible packages can be installed in the same system.

To Thierry's point about newer distro's, there are distro's today starting to form around Docker as a packaging device and it does not have the same issues that traditional distro's do. Fedora/Redhat Atomic, CoreOS, RancherOS are some examples. You can run incompatible rabbit's on the same server. Both can be patched to the latest secure version, but simply incompatible with each other. Say a stable v1 branch and a stable v2 branch. They probably share every package except 1, and at a file system level actually do share all the space but the change.

The most interesting bit about containers to me, is that the traditional distro's are often used as source materiel for the container images. So it doesn't eliminate debs/rpms. it just puts a wall around them to make conflicts not so bad.

There's a whole nother discussion about keeping the containers up to date.... its a hard problem and not well solved yet. But I do think that's fixable. Most likely the traditional distro's will test/ship containers based on their traditional distro and refresh the containers when any of the deps get updated too. This allows the distro's to continue to do the job that they are already great at. Packaging up, testing, and releasing great software that as a user, you can trust.

Thanks,
Kevin


________________________________________
From: Thomas Goirand [zigo at debian.org]
Sent: Tuesday, April 19, 2016 3:16 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [release][requirements][packaging][summit] input needed on summit discussion about global requirements

On 04/19/2016 11:48 AM, Thierry Carrez wrote:
>> Remember that in distros, there's only a single version of a library at
>> any given time, at the exception of transitions (yes, in Red Hat it's
>> technically possible to install multiple versions, but the policy
>> strongly advocates against this).
>
> Depends on the distro. Gentoo for example lets you have multiple
> versions of libraries installed at any given time.

Earlier, I wrote that Gentoo was probably an exception, but reading
posts from Matthew Thode, he explained that it wasn't really a good idea
even in Gentoo to bundle libs. So let's consider that's not an option
for all distros packaging OpenStack.

> I expect in the future the one-version-of-lib-at-a-time
> will more and more be seen as a self-imposed limitation

It has always been the case that it's a self-imposed limitation, because
that's the only sane way to maintain software.

> and less as a
> distribution axiom, and next-generation distros will appear that will
> solve that limitation one way or another.

I very much doubt that distribution will attempt shooting themselves in
the foot, and allow software to use deprecated, old versions of libs,
which conflict with each other. What can be done is mitigate issues, at
most. I fail to see how hiding the issue that 2 libs are conflicting, or
that component A can't support the latest version of component B can be
seen as the "next-generation" thing that solves all troubles.

I'm very much concerned that we're here, taking the same wrong approach
of thinking that containing libs and avoid conflict will solve troubles.
That's not the case, it just hides it. We still need to make sure that
components will support the last versions of everything at the release
time, otherwise, we'll have serious issues maintaining stable branches.

If the problems is just releasing a lot of work from the release team
(and infra, as you just pointed, TTX), then I have given a list of
things we could do. These things wont IMO impact quality, and will still
allow co-instability. I'll attempt to list it again, in a more clear way:

1- stopping to propagate the global-requirements.txt versions *of
OpenStack components* to each and every project: this isn't helping
anyone anyway, since both the gate and downstream distros are going to
use the latest version.

Example:
Let every component decided what is the minimum version of oslo.utils
that it needs, but still force it to support the latest
upper-constraints.txt version.

2- stop maintaining requirements for modules which are only needed by a
single project. Simply listing it (to make sure we don't have a 2nd
component that adopts it without knowing about the first) will be
enough, and this could be done in a separate file, which would point at
the project.

Example:
Take PuLP out of global-requirements.txt, and move it to "used-by.txt"
(the name of that file can be something else):
echo "PuLP: congress" >>used-by.txt

Then let Congress decide what version it needs.

I'm not sure yet about the consequences for 3rd party libs, this will
probably require more thinking and brainstorming, but right now, it's my
strong believe that we still need to maintain key packages in the
global-requirements.txt (I'm thinking about Alembic, SQLA and such).

Would the above releasing a lot of work from both infra and release
teams? Is this "enough", or would there still be too much work? Maybe we
could give this new workflow a try for a cycle, and see how it works
out? Please, voice your concerns, share your view, etc.

Hoping the above is constructive enough,

Cheers,

Thomas Goirand (zigo)


__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev