Open Stack

Helllo,

On Thu, Dec 9, 2021, at 19:03, Jeremy Stanley wrote:
> The reason I raised those points on the review was in order to
> highlight inconsistencies with OpenStack as a whole. If OpenStack is
> defined by anything, it's consistent development workflows and
> automation, which allow small groups of people to take advantage of
> economies of scale in order to minimize maintenance over time across
> the whole of the set of projects which comprise it. Inconsistency
> has an inherent cost which the entire community bears, from tracking
> integration failures in development to keeping concise recipes for
> packaging in distributions to operators being able to configure all
> services the same way and expect the same logging formats.

I agree that consistency is important, and helps us in many ways.
I don't  think it's the "defining" factor of OpenStack, however.
Inconsistency in openstack is in many places, from the users' to the developer's perspective.
It doesn't mean I am _looking for_ inconsistency.
I am merely embracing the differences (and I am eager to remove them, if we can).

> Most of the benefits a project gets from being "part of OpenStack"
> are derived directly from the established standards and norms which
> enable OpenStack to be treated as a cohesive whole rather than a
> mere "bucket of parts" with tooling choices made independently by
> their respective development teams. Deviation from these
> expectations should be deliberate and come with a clear rationale
> which benefits OpenStack as a whole rather than one project alone.

Correct again! :)

For the case of skyline, one could made a case to start building packages with a new way, using poetry, rather than setuptools. It could mean we need to ask the skyline team to help on improving the infrastructure and testing.
Instead of arriving to a standard testing, we embrace the difference and provide a possible new way of working. We could have two kinds of python projects. Instead of accepting that difference when a single project asking for it, maybe we can just tally who wanted this, and the TC decides at the end "when" it is worth doing.

My whole point is that we need to document better and agree which parts benefits OpenStack as a whole.
We should also have indicators for checking how the "ecosystem" is behaving. If we are considering the amount of new teams in OpenStack, it's not very encouraging. (It might not be a metric, I don't know, I rather leave Thierry judge there).

In other words:  Is it really better to change the skyline team to adapt our mold, or should we have multiple molds?
We agreed to go the way that skyline should adapt. I am simply raising the fact that some other people/projects won't do the effort to adapt to existing mold, because they won't necessarily see the value.
(I won't prevent my teams to work on non-openstack projects ;))

> Let's take the current example point by point:
>
>   * Skyline implements its own routines for things like
>     configuration and logging rather than using Oslo libraries.
>     Should all OpenStack projects "move forward" to this model,
>     choose their own configuration and logging formats, implement
>     their own solutions for documenting these, and so on? Does
>     inconsistency here benefit operators in some way?

Is Oslo a requirement to be an OpenStack project now?
AFAIK it wasn't (see swift).

It doesn't mean you can't be consistent for logs/configuration from the user perspective by not using oslo.
Side note: Bringing oslo projects for configuration and logging isn't enough to bring consistency from the user perspective.

>   * Skyline defines the interdependencies between its components
>     with a mix of "monorepo" (multiple packages from a single
>     repository) and Git submodule references (pinning specific
>     commit ids as virtual subtrees of another repository), rather
>     than establishing a loose coupling with explicit version numbers
>     and package relationships. This choice prevents them from taking
>     advantage of features in Zuul like cross-project change
>     dependencies. Is this a pattern the other OpenStack projects
>     should follow?

Which is why I am highlighting that _as of today_, the governance requires certain expectations, which needs clarifications (or exceptions) if we want to integrate Skyline.

I know I am a bit pedantic in the following question:
Is (nowadays) the expectation for projects to be testable by Zuul (+release-able by zuul, ... list to be defined) or is the expectation for projects to be following the PTI guidelines (tested by zuul)?

In other words: Do we need to enforce the tooling or the results (a frequently released and well tested software)? 
I would like the TC to think about this.

>  Does it benefit distribution package maintainers
>     to need to track fundamentally different repository layouts and
>     build processes for each of them?

Isn't that exactly what the distros are doing right now with the packages they are maintaining? Not every software is like openstack. Yes, almost all openstack projects are similar to each other, so they can be packaged the same way.

Yet, some official openstack projects aren't, as of today. So, some distros decide to not package them? Is that a problem for us? I didn't think so. Projects/Teams need to understand that if they don't follow "x", they will not have "y". Which sounds fair to me.

Keep in mind that we are _already doing exceptions_. Not all projects are similar to nova, glance, neutron, keystone, cinder. (Sorry for the many I missed).

>   * Skyline performs its release versioning by committing the
>     version strings to files in its repositories rather than
>     assigning them at build time from Git tag metadata. This leads
>     to ambiguity as to what exact state of the repository represents
>     that version, as well as opening up the possibility of
>     forgetting to merge the version update before tagging (a very
>     common problem which drove OpenStack to rely on its current
>     model). Wholly different tools are also needed in order to
>     track versions for these projects as a result. Should the rest
>     of OpenStack's projects follow suit there? Has something changed
>     to make the idea of using Git to signal releases a bad one?

You're raising again a valid point, for which the skyline team will have to find a solution.

For me, it boils down to defining that projects should follow a "project release interface", which explains the expected behaviour in regards of releasing for projects. We don't have such. I am not claiming we should, I am merely asking us to think about what we are doing.

Projects could indeed move to use pbr/setuptools. However, I am not sure many developers understand why.
Should we simply explain what is expected in terms of building the version from git? People might even come with interesting solutions. Promoting creativity will create tech debt, but might also reduce it over time.
What if we don't need pbr anymore? Will we even notice?

>   * Skyline uses makefiles to script its build and test workflows
>     rather than tox, adding a tox wrapper around the make targets
>     in order to nominally comply with the OpenStack PTI. As a
>     result, it's unable to rely on most of tox's features for
>     environment isolation, package management, path overrides, et
>     cetera. OpenStack projects have solved a lot of cross-project
>     development workflow challenges through applying consistent
>     changes across the tox.ini files of repositories, should those
>     solutions be abandoned and recreated for make instead?

I didn't say so. I am only partially agreeing with you there, for once :)

For me, tox is a test runner that provides many nice features thanks to its code (and integration with setuptools)
I would prefer if python projects using tox for testing interface indeed.
Should that be the standard? I don't know.
Does it make sense to rely on tox features for isolations/package management/other? I don't know.
You might be aware that it's possible to integrate tox with poetry. Does it provide all we _need_? I don't know: We aren't explicit in the requirements of what we need, we just talk about tools.

If I were to rephrase: Is it easier to fit the existing mold or clarify what we need, why, and how we can deal with the tech debt? Probably the former. Should we do it? I don't know. The TC should however evaluate this seriously.

>   * Skyline does not put its external Python package dependencies in
>     central lists nor use tools which would allow it to rely on
>     OpenStack's global constraints implementation, making it very
>     hard for them to guarantee that their software can run with the
>     same versions of shared dependencies as the rest of OpenStack.
>     Is this unnecessary? Have we given up on the idea that two
>     pieces of OpenStack software can be coinstalled into one
>     environment, or that Linux distributions won't be stuck having
>     to package many different versions of the same software because
>     OpenStack projects don't agree on which versions work for them
>     any longer?

Now, we are hitting the nail, IMO :)

This is a bit of the issue's core, isn't it? I think that's what we need to document.
While I (personally) don't believe it's important nowadays (cf. containers), I am sure distros will disagree with me.
It's important to know who we are pleasing, as we can't please everyone ;)
Let's be clear about the audience.

If we intend to please the distros, did we ask about RedHat/Canonical committments on skyline? Are they even remotely interested? If they want to use it, wouldn't it make sense they do the work to make sure skyline is packageable for them, by improving upstream? If they don't, why do we care?

> To be perfectly clear, I don't oppose accepting Skyline as a project
> in OpenStack, even in its current state, but I do feel like it's not
> going to end up released as part of OpenStack unless it can address
> the challenges I outlined above. Most of these are fairly
> superficial changes to implement, however, and can be done by simply
> copying the solutions already used in other OpenStack projects or
> retroactively applying our cookie-cutters.

I agree with you.
I am merely asking for clarity. However, I want to think as a whole.

> "Moving OpenStack
> forward" to be more like Skyline, on the other hand, is bound to be
> far more work than the Skyline team can handle on its own and take
> far longer, though I'm not opposed to the idea of discussing which
> of these differences may make sense for OpenStack to adopt in the
> future.

Depends on the definition of "Moving OpenStack forward".
Do you mean all modifying all projects to be like skyline + integrate the testing? That was never my point.
Do you mean "ensure a single project can be behaving like a majority of other big openstack projects are"? That's indeed more work for a team, compared to if they had their work done outside OpenStack governance.

Should we use this energy for thinking(/doing something) about OpenStack future? IMO, absolutely.

And for those arriving here: Thanks for reading my rant until the end ;)
Jean-Philippe Evrard (evrardjp)