Open Stack

As part of the push to release code from the oslo incubator in
stand-alone libraries, we have had several different discussions about
versioning and release schedules. This is an attempt to collect all of
the decisions we have made in those discussions and to lay out the
rationale for the approach we are taking. I don't expect any of this
to be a surprise, since we've talked about it, but I haven't actually
written it all down in one place before so some of you might not have
seen all of the points. Please let me know if you see any issues with
the proposal or have questions. If everyone agrees that this makes
sense, I'll put it in the wiki.

Doug



Background:

We have two types of oslo libraries. Libraries like oslo.config and
oslo.messaging were created by extracting incubated code, updating the
public API, and packaging it. Libraries like cliff and taskflow were
created as standalone packages from the beginning, and later adopted
by the oslo team to manage their development and maintenance.

Incubated libraries have been released at the end of a release cycle,
as with the rest of the integrated packages. Adopted libraries have
historically been released "as needed" during their development. We
would like to synchronize these so that all oslo libraries are
officially released with the rest of the software created by OpenStack
developers.

The first release of oslo.config was 1.1.0, as part of the grizzly
release. The first release of oslo.messaging was 1.2.0, as part of the
havana release. oslo.config was also updated to 1.2.0 during havana.
All current adopted libraries have release numbers less than 1.0.0.

In the past, alpha releases of oslo libraries have been distributed as
tarballs on an openstack server, with official releases going to PyPI.
Applications that required the alpha release specified the tarball in
their requirements list, followed by a version specifier. This allowed
us to prepare alpha releases, without worrying that their release
would break continuous-deployment systems by making new library
releases available to pip. This approach still made it difficult for
an application developer to rely on new features of an oslo library,
until an alpha version was produced.

When the PyPI mirror was introduced in our CI system, relying on
tarballs not available on the mirror conflicted with our desire to
have the gate system install *only* from the package mirror. As we are
now installing only from the mirror, we need to publish our alpha
releases in a format that will work with the mirror.

We already gate OpenStack applications and oslo libraries using
integration tests using the normal devstack-gate jobs. During Icehouse
we had a couple of oslo library releases that broke unit tests of
applications after the library was released. We plan to address that
with separate gating jobs during juno. In addition to that gating, we
need to support developers who want to use new features of oslo
libraries before official releases are available.

A Version Numbering Scheme:

At the Juno summit, Josh proposed that we use semantic versioning
(SemVer) for oslo libraries [1]. Part of that proposal also included
ideas for allowing breaking backwards compatibility at some release
boundaries, and I am explicitly *not* addressing
backwards-incompatible changes beyond saying that we do not expect to
have any during Juno. We do need to solve the problem of breaking API
compatibility, but I want to take one step at a time. The first step
is choosing a rational release versioning scheme.

SemVer is widely used and gives us relatively clear guidelines about
choosing new version numbers. It supports alpha releases, which are
going to be key to meeting some of our other requirements. I propose
that we adopt pbr's modified SemVer [2] for new releases, beginning
with Juno.

The versions for existing libraries oslo.config and oslo.messaging
will be incremented from their Icehouse versions but updating the
minor number (1.x.0) at the end of the Juno cycle.

All adopted libraries using numbers less than 1.0 will be released as
1.0.0 at the end of the Juno cycle, based on the fact that we expect
deployers to use them in production.

Releases during Juno should *all* be marked as alphas of the
anticipated upcoming SemVer-based release number (1.0.0.0aN or
1.2.0.0aN or whatever). The new CI system can create packages as
Python wheels and publish them to the appropriate servers, which means
projects will no longer need to refer explicitly to pre-release
tarballs.

Releases after Juno will follow a similar pattern, incrementing the
minor number and using alpha releases within the development cycle.

[1] https://etherpad.openstack.org/p/juno-oslo-semantic-versioning
[2] http://docs.openstack.org/developer/pbr/semver.html

Frequent Alpha Releases:

While we can run gate jobs using the master branch of oslo libraries,
developers will have to take extra steps to run unit tests this way
locally. To reduce this process overhead, while still ensuring that
developers use current versions of the code, we should produce alpha
releases of libraries during the release cycle fairly frequently. I
suggest that we start with a weekly check-up during the oslo team
meetings, and that we tag releases early on Mondays when deemed
necessary.

Patch Releases:

Updates to existing library releases can be made from stable branches.
Checking out stable/icehouse of oslo.config for example would allow a
release 1.3.1. We don't have a formal policy about whether we will
create patch releases, or whether applications are better off using
the latest release of the library. Do we need one?

We do not typically use upper bounds on the requirements
specifications for oslo libraries, so new releases may automatically
be adopted by continuous-deployment systems building packages for
stable branches of OpenStack applications. Although we have been
careful about API compatibility in the past, there is a chance that a
new release could break an older application. Applications could add
an upper bound using SemVer numbering if they choose, although that
may prevent them from seeing bug fixes.

Unit test gate jobs:

We have a blueprint for Juno to add cross-project unit test gating for
applications and oslo libraries [3]. This will allow us to verify that
tests for applications do not break when oslo libraries change, but
also that those tests do not make assumptions about oslo library
implementation details.

Depending on how the job is implemented, we may be able to reuse the
tools to let developers run the same tests locally. More thought is
needed here; stay tuned.

[3] https://blueprints.launchpad.net/oslo/+spec/enhance-cross-test-gate-job