Open Stack

On 2021-12-09 16:57:27 +0100 (+0100), Jean-Philippe Evrard wrote:
[...]
> So the question is: Is OpenStack ready to move forward and adapt
> to new kinds of packaging/software? If yes, let's modify the PTIs
> (or python PTI) to cover the case for skyline, and allow that team
> to move forward. It's not like it's doing something very different
> than some new python projects.
> 
> Yes, OpenStack is mature... I (simply) hope it's also a place for
> casual contributions, and where developers can have fun with tech.
> And yes, it won't please everyone from day one (especially
> distros), but I hope we can work this out together, rather than
> discard the project because it doesn't fit the mold.
[...]

The reason I raised those points on the review was in order to
highlight inconsistencies with OpenStack as a whole. If OpenStack is
defined by anything, it's consistent development workflows and
automation, which allow small groups of people to take advantage of
economies of scale in order to minimize maintenance over time across
the whole of the set of projects which comprise it. Inconsistency
has an inherent cost which the entire community bears, from tracking
integration failures in development to keeping concise recipes for
packaging in distributions to operators being able to configure all
services the same way and expect the same logging formats.

Most of the benefits a project gets from being "part of OpenStack"
are derived directly from the established standards and norms which
enable OpenStack to be treated as a cohesive whole rather than a
mere "bucket of parts" with tooling choices made independently by
their respective development teams. Deviation from these
expectations should be deliberate and come with a clear rationale
which benefits OpenStack as a whole rather than one project alone.

Let's take the current example point by point:

  * Skyline implements its own routines for things like
    configuration and logging rather than using Oslo libraries.
    Should all OpenStack projects "move forward" to this model,
    choose their own configuration and logging formats, implement
    their own solutions for documenting these, and so on? Does
    inconsistency here benefit operators in some way?

  * Skyline defines the interdependencies between its components
    with a mix of "monorepo" (multiple packages from a single
    repository) and Git submodule references (pinning specific
    commit ids as virtual subtrees of another repository), rather
    than establishing a loose coupling with explicit version numbers
    and package relationships. This choice prevents them from taking
    advantage of features in Zuul like cross-project change
    dependencies. Is this a pattern the other OpenStack projects
    should follow? Does it benefit distribution package maintainers
    to need to track fundamentally different repository layouts and
    build processes for each of them?

  * Skyline performs its release versioning by committing the
    version strings to files in its repositories rather than
    assigning them at build time from Git tag metadata. This leads
    to ambiguity as to what exact state of the repository represents
    that version, as well as opening up the possibility of
    forgetting to merge the version update before tagging (a very
    common problem which drove OpenStack to rely on its current
    model). Wholly different tools are also needed in order to
    track versions for these projects as a result. Should the rest
    of OpenStack's projects follow suit there? Has something changed
    to make the idea of using Git to signal releases a bad one?

  * Skyline uses makefiles to script its build and test workflows
    rather than tox, adding a tox wrapper around the make targets
    in order to nominally comply with the OpenStack PTI. As a
    result, it's unable to rely on most of tox's features for
    environment isolation, package management, path overrides, et
    cetera. OpenStack projects have solved a lot of cross-project
    development workflow challenges through applying consistent
    changes across the tox.ini files of repositories, should those
    solutions be abandoned and recreated for make instead?

  * Skyline does not put its external Python package dependencies in
    central lists nor use tools which would allow it to rely on
    OpenStack's global constraints implementation, making it very
    hard for them to guarantee that their software can run with the
    same versions of shared dependencies as the rest of OpenStack.
    Is this unnecessary? Have we given up on the idea that two
    pieces of OpenStack software can be coinstalled into one
    environment, or that Linux distributions won't be stuck having
    to package many different versions of the same software because
    OpenStack projects don't agree on which versions work for them
    any longer?

To be perfectly clear, I don't oppose accepting Skyline as a project
in OpenStack, even in its current state, but I do feel like it's not
going to end up released as part of OpenStack unless it can address
the challenges I outlined above. Most of these are fairly
superficial changes to implement, however, and can be done by simply
copying the solutions already used in other OpenStack projects or
retroactively applying our cookie-cutters. "Moving OpenStack
forward" to be more like Skyline, on the other hand, is bound to be
far more work than the Skyline team can handle on its own and take
far longer, though I'm not opposed to the idea of discussing which
of these differences may make sense for OpenStack to adopt in the
future.

My deeper concern is that these differences are a clear indication
that the software was written outside the OpenStack community, with
no original consideration for consistency with OpenStack itself, and
so there are probably more significant architectural modifications
needed (beyond integrating oslo.config and oslo.log) which I'm not
well-placed to spot. Those can addressed in time too, and while
Skyline is an official team within OpenStack, but they will still
need to be taken care of at some point early in the life of the
project either by becoming like the rest of OpenStack or helping the
rest of OpenStack become more like Skyline.
-- 
Jeremy Stanley
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 963 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20211209/aed49b6d/attachment.sig>