[openstack-dev] [stable] Preparing 2014.2.4 (Juno) WAS Re: [Openstack-operators] [stable][all] Keeping Juno "alive" for longer.

Jeremy Stanley fungi at yuggoth.org
Wed Dec 2 16:51:24 UTC 2015

On 2015-11-17 02:49:30 +0000 (+0000), Rochelle Grober wrote:
> I would like to make a plea that while Juno is locked down so as
> no changes can be made against it, the branch remains on the
> git.openstack.org site.

We don't so much delete the stable/juno branch as replace its final
state with a tag called juno-eol. It's a trivial Git operation
downstream to recreate your own stable/juno branch from our juno-eol
tag and continue using it however you like. This is simply our way
of indicating clearly that we won't ever be adding new patches to
stable/juno (one might say that makes it _extremely_ stable).

> Please? One area that could be better investigated with the branch
> in place is upgrade. Kilo will continue to get patches, as will
> Liberty, so an occasional grenade run

TL;DR: no.

While I can understand the desire, the reality is that we stop doing
upgrade testing from unsupported branches for very practical
reasons. In our CI, Grenade relies on environments built by
DevStack. To test an upgrade from Juno to Kilo, you first need
DevStack working on Juno. I know it probably doesn't make a lot of
sense that we would suddenly cease to be able to install a version
of DevStack which worked at one point in time, but there are two
primary factors that cause this to be the case: instability within
the Python packaging ecosystem, and system support requirements.

Our DevStack deployments rely on pip to install Python dependencies
of OpenStack from PyPI. The way we traditionally declared these
dependencies was to track loose or even open-ended ranges of version
numbers in an attempt to keep our testing as current with the state
of those dependencies as possible. This means that after a time, the
behavior of some of those dependencies may change in such a way that
new versions we expect to work with OpenStack in fact do not. Given
that our dependencies number in the hundreds and have for a while,
and that our direct dependencies also in turn often declare loose or
open-ended version ranges for their own dependencies (our transitive
dependencies), this breaks down so often that we're usually unable
to start DevStack in our CI by the time we reach the EOL timeframe
for a given branch.

OpenStack also has (direct and indirect) dependencies outside of
Python, such that it's frequently impossible to run DevStack on a
non-contemporary Linux distribution. As a result, our CI ties
OpenStack versions to specific Linux distro release series, and so
continuing to support installing a given version of OpenStack
requires our CI maintainers to continue supporting Linux distro
releases contemporary with those which existed suring the
development cycle leading up to the initial version of a given
stable branch. For example, we maintained Ubuntu 12.04 as a test
platform through the lifetime of stable/icehouse because that's what
was current at the start of the Icehouse development cycle. We're
maintaining CentOS 6 as a test platform through the end of
stable/juno for similar reasons. The more stable branches we commit
to supporting, the more different distro releases our Project
Infrastructure team ends up being on the hook to maintain.

The Python packaging ecosystem situation is improving, and during
the Liberty development cycle OpenStack also grew the beginnings of
some new mechanisms to more accurately reproduce the previously
known-working conditions under which releases were developed and
tested. Reproducibility of DevStack operation has much greater
long-term promise for the stable/liberty branch, and by extension
upgrade testing with Grenade should be able to take advantage of
that for the stable/mitaka branch. However, the long and short of it
is that to support upgrade testing we need to (at least to some
extent) support the version from which the test is upgrading. This
is contradictory with performing upgrade testing *to* every
supported stable branch, since there will always be a starting point
we test upgrading *from* without being able to continue supporting
the version which came before it.
Jeremy Stanley

More information about the OpenStack-dev mailing list