Hello Jay, Jeremy, all,

thanks for your responses and the time to write up your thoughts.


On 14.02.24 14:53, Jeremy Stanley wrote:
Don't lose sight of the fact that this is an inherently existential
question. OpenStack is all of us. You are OpenStack. I am OpenStack.
Why are you not pushing ahead? Why am I not pushing ahead? This is
the real question you're asking.
That is exactly on point to why a discussion as a starting point can be important to "push ahead".
While I might be of some opinion that things are not as they could or should be, it might not be the consensus of the greater community, other important and hard-working members of this community.
Does it really help if everybody forms their ideas right into a patch blackening all Python code of a project he once wrote a little patch for?
Or does it help the generals idea if O start sprinkling type annotations all over any code that is OpenStack?
Who would review or even accept that? Writing up a quick proposal requesting the use of type annotations and pylint for all OpenStack code from now on might sound nice, but how would that come across? I know we are using Python, which asks for forgiveness instead of permission, but really?
What we need for any change, apart from any policy or format agreement is foundation is the intention to change, an in a certain direction.

A discussion to me is a good way to see if your idea is onto something and if others might even share it (seems to be the case in the matter of modernizing Python).
It's also a good way to find others that might work together on the next step: A formal proposal if e.g. someone from TC even encourages to do that :-)
But even if that proposal was then not even written by OP or other participants of the discussion - a heated discussion or the agreement by some established OpenStack contributors, might even be a good indication to TC that something is an important topic which could need some formal backing or follow up in one form or another. 
What else would be a good source for TCs agenda than the combined voices of the community?



On 14.02.24 17:21, Jay Faulkner wrote:
> The community has been very capable if an external change (Death of
> Python 2.7, SQLAlchemy 2.x, eventlet EoL, ....)
> forced some reaction or adaption. But does it really require some
> external force for some kind of modernization?

I'm isolating this for a specific reason: many of the items listed here were driven by the same major contributors (human and corporate) to the community, and/or are in progress. For instance, the eventlet migration still does not have a finalized plan[0] and the primary drivers right now are developers from RedHat as well as a single GR-OSS developer we contributed to help improve eventlet in the short term. When describing this to my boss; I told him I'm incredibly proud that GR-OSS is participating in this work, because so few contributing companies are willing to do this invisible work -- but the sad thing is, eventlet eol is one of a large number of things that needs doing -- and there aren't many people to do it.

I understand your frustration regarding the work on eventlet and yes, these kind of components and libraries are mostly taken for granted. This was seen quite a few times in recent years when e.g. the log4j issue happened.

I simply picked a few examples, where OpenStack, and I am referring to the community of projects and people, to my impression ran into an issue or a deprecation and there was lots of energy to make the required changes happen across many projects quickly to come out better on the other end, technically and socially. But more importantly it was the challenge and need for action that brought people together in accepting change. While you personally might rightly complain that the urgently needed work on eventlet was only done by a few, I also see the consensus for the necessary or even overdue change as another outcome.

To maybe try another example, Python 3.12, e.g. https://review.opendev.org/q/topic:%22python-3.12%22
While frustrating to see a simple version bump of Python causing lots of issues and CI fails. Put in the right perspective, any of those changes is part of that push forward and that keep the codebase alive.


This email thread highlights many of those other issues which need addressing; but are missing a significant portion: a call to action and real work to make them happen.

There are several ways you help change this:
1) Don't like how the project is being run from a technical perspective? Run for the TC -- there certainly is a need for volunteers. Try to make change happen. Governance is not easy and there are hundreds of people with different opinions about what direction "forward" is -- and it's a lot more work than just adding and removing python versions from platform documents.

Not to repeat what I said above. But there might be hundreds of opinions voiced here and otherwise. What's important is a good mannered discussion leading to some kind of distillation of opinions into somewhat of an agreement, a common denominator. And isn't the open and public discussion actually helpful for TC identify an issue that is worth looking into? Why should one start with a nicely written up proposal holding all the answers, instead of starting a discussion around a topic first and then see if others share the impressions that the Python stack has some issue that really should be tackled?


2) Do a proof of concept in a smaller project. I'd personally be willing to help you gain consensus on implementing (for instance) some type hints on Ironic. I can't promise the whole Ironic team would agree; but the point is the easiest way to influence change in OpenStack is to show willingness to do the work. For SQLA 2.0 migration, for instance, it was obvious there were some members of the community who worked towards it, with urgency, even pulling forward projects that are now inactive to function with SQLA 2.0.

Don't get me wrong, but I feel there is a little more of general frustration than actual reasoning here. Discrediting the (unknown) amount or value of contributed work by someone else, likely is not going to motivate them to help. Reading this, I was just about to vent on my own frustration of trying to get even some little patch reviewed and merged and the need drive even small changes home over multiple releases. This contributes, just like the technical state of the Python stack, to developer experience, their willingness to contribute and stay with the project. And I'd like to generalize this as much as I can, as it's quite universal for my experience with different OpenStack projects and not related to any person or team in particular. But having contributions (and also bug reports) that are not responded to in any way will scare people away from contributing and likely for good.

I gladly write up more detailed thoughts and experiences if someone would be using them to reflect on how things are organized.


3) Advocate inside your organization for more upstream contribution; either in the form of your time or additional upstream contributors. A recent paper[1] out of Harvard Business School says 96% of the demand-side value is created by only 5% of OSS developers. Everyone who has a leadership position at a company that uses or contributes to OSS has an obligation to advocate to help close these gaps.

While I agree with this point in general, again, I don't think it's a good response to the topic of this thread.
There are many issues, big and small, that contribute to the fact that OpenStack receives less contributions over time and most of them are by a small group of people, most of them on the payroll of the same group of companies. While I agree that institutions and companies should do their part, also asking why OpenStack and it's projects are not really attractive to contribute to as a sole developer from some smaller installation might also be helpful to discuss as it ensures the community remains healthy and broad. Yet another good topic for governance. Not to answer, but to bring to the agenda and actively address.


I'm not going to break down the original email point by point, but generally: yes, there are things that OpenStack are behind on -- but emails to the mailing list and consensus aren't the solution. The solution is more people contributing to solve these problems -- especially in traditionally under-resourced teams such as QA, oslo libraries, and smaller projects on the verge of inactivity. Jeremy said (paraphrased) that we are OpenStack; rather than asking why OpenStack is not pushing forward, we should be asking ourselves why we aren't pushing forward. I agree with him. I also agree with the OP which asserts that we have clear technical improvement opportunities. What's stopping you from pushing them forward?

What made you assume the OP or myself are not actively pushing specs, code and ideas (like this one here)? Did you consider the urge to post to the ML and participate in a discussion on modernizing the Python stack could potentially be out of the experience doing so? Even if I hadn't done any such work, would the arguments about the Python stack be less valid?


If I may bluntly and selectively quote https://review.opendev.org/c/openstack/governance/+/902585/16/goals/proposed/migrate-eventlet-usages-to-asyncio.rst#1075 ....

A community goal does not shape a new and personal vision of Openstack.
A Community goal collects this vision from the scattered hopes and intentions of our community's past.
[...]
The TC is elected to provides technical leadership.
[...]
Without action we will soon face a brutal discontinuation.


With that, let me sum up my thoughts a little ...

While some tech debt in the Python stack is not an immediate existential crisis, it's vital to keep moving and to stay on top of things.
Adopting new developments in the ecosystem is for the sake of quality (security, bugs, speed, efficiency, ...) and to avoid running into short term firefighting mode (deprecation, technical dead-ends, ...). But most importantly, if I read your responses, Jay and Jeremy, it's about lowering the barrier for "outside" contributions again.
Code has to be approachable (e.g. is well structured, has type hinting, current syntax styles, ...) by part-time contributors to even have them try contributing and push a patch. The mentioned well established tooling such a formatters, linters or static code analysis and others should be used to assist senior and full-time devs with the QA and review processes to ultimately enable more contributions and by more individual contributors to be merged.



Thanks again for your time,


Christian