New Openstack Deployment questions - how will we respond to this change?

Alex Schultz aschultz at
Tue Dec 15 15:00:46 UTC 2020

On Tue, Dec 15, 2020 at 7:46 AM Braden, Albert
<C-Albert.Braden at> wrote:
> The purpose of my Openstack Deployment email was not only to rant about the decision to kill Centos 8. I want to make sure that we don't blindly go down the easy route of pretending that Centos Stream is an acceptable replacement for Centos. If we do this, we would do a disservice to the community and to everyone who depends on it. The reality is that, at the end of 2021, Centos will no longer exist as a viable production operating system, and everyone who was using it will have to adjust. For companies that already have Centos mirror repositories setup and people managing them, the increased load imposed by Stream will be incremental, but I suspect that the vast majority of companies do not have that infrastructure already setup, and we will need to carefully consider whether our needs are better met by going forward with Stream or switching to another OS.

>From an OpenStack development perspective, centos8 stream is an
acceptable centos replacement because it finally allows us to get a
head of "stable" versions and get the issues resolved early and get
fixes committed upstream faster. It may not be an acceptable
replacement for operators, but that's up to the operator to determine.
An operator can delay updates from stream and implement their own
rolling updates as necessary via CI/CD (via repo mirroring) which
previously was not the case when CentOS dropped exposing specific
minor point releases. Since the inception of 8, there hasn't been an
8.1 or 8.2 you could pin to. You got rolling updates whenever 8.2 hit,
you got it without being able to go back.

> If the OpenStack community decides to continue building on Stream, we should make it crystal clear to operators and users that Stream is not a production-ready OS and that our Stream OpenStack implementation is suitable for testing and development use only, unless they devote substantial resources to mirroring and testing Stream to insulate production clusters from the instability that it will introduce.

The determination of a 'production-ready' OS is likely not something
that OpenStack should be expressing.  IMHO, that decision is an end
user decision based on their comfort for the risk involved. From an
RDO/TirpleO perspective, we will be aligning on CentOS Stream going
forward so we will be addressing issues as they come up. We will
likely need to improve our documentation around supported module
versions, etc.

> If we do decide to continue building on Stream, how will we respond to the constant stream of bug reports that will result when Stream changes cause OpenStack components to fail? Many component teams are already understaffed. Will we have the bandwidth to constantly chase changes? Should we replace our Centos builds with RHEL, or with Rocky? Does the community have (or can we find) the resources to do the work of maintaining stable Stream mirrors and only building OpenStack on our stable versions of Stream? Or would it be better to drop Centos support and focus our efforts on operating systems that have not implemented unilateral changes that harm the community?

TBH I don't think things change. We've always had to respond to
breaking versions even with CentOS.  We just hit a fair number of
issues with 8.3 being released last week.  I think this actually
reduces the impact because we're likely to get individual failures
that can be identified/addressed instead of large sweeping changes
that we get when there's an 8.2 -> 8.3 transition.

> -----Original Message-----
> From: Braden, Albert
> Sent: Friday, December 11, 2020 9:10 AM
> To: openstack-discuss at
> Subject: RE: [EXTERNAL] Re: New Openstack Deployment questions
> Centos Stream is fine for those who were using Centos for testing or development. It's not at all suitable for production, because rolling release doesn't provide the stability that production clusters need. Switching to Centos Stream would require significant resources to be expended to setup local mirrors and then perform exhaustive testing before each upgrade. The old Centos did this work for us; Centos was built on RHEL source that had already been tested by paying customers, and bugs fixed with the urgency that paying customers require.
> Adding an upstream build (Stream) to the existing downstream (Centos 8.x) was fine, but I'm disappointed by the decision to kill Centos 8. I don't want to wax eloquent about how we were betrayed; suffice it to say that even for a free operating system, suddenly changing the EOL from 2029 to 2021 is unprecedented, and places significant burdens on companies that are using Centos in production. I can understand why IBM/RH made this decision, but there's no denying that it puts production Centos users in a difficult position.
> I hope that Rocky Linux [1], under Gregory Kurtzer (founder of the Centos project) will turn out to be a useful alternative.
> {1}
> -----Original Message-----
> From: Luigi Toscano <ltoscano at>
> Sent: Thursday, December 10, 2020 3:51 PM
> To: Thomas Wakefield <dwakefi2 at>; openstack-discuss at
> Cc: Satish Patel <satish.txt at>
> Subject: [EXTERNAL] Re: New Openstack Deployment questions
> CAUTION: The e-mail below is from an external source. Please exercise caution before opening attachments, clicking links, or following guidance.
> On Thursday, 10 December 2020 15:27:40 CET Satish Patel wrote:
> > I just built a new openstack using openstack-ansible on CentOS 8.2
> > last month before news broke out. I have no choice so i am going to
> > stick with CentOS.
> >
> > What is the future of RDO and EPEL repo if centOS going away. ?
> Continue as before on CentOS Stream.
> --
> Luigi
> I apologize for the nonsense below. So far I have not been able to stop it from being attached to my external emails. I'm working on it.
> The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.

More information about the openstack-discuss mailing list