[tc] dropping python 3.8 support for 2024.1
I thought about this issue some more after yesterday's TC meeting, and tried to articulate why I think keeping python 3.8 support for 2024.1 is a bad idea on the patch: https://review.opendev.org/c/openstack/governance/+/895160 I don't know if my argument there will change anyone's mind, but since a lot of people have already voted on the gerrit review, I wanted to make sure that people are at least aware of it, to wit: I just don't see that py38 support for 2024.1 makes sense. It's not a default version in Ubuntu 22.04, Debian 12, Debian 11, CentOS Stream 9, or Rocky Linux 9, which are the distros specifically called out in the current 2024.1 PTI that this proposal is patching. While we can use Ubuntu 20.04 to run unit tests, we can't run master (2024.1 development) devstack in it, so it doesn't seem to me that Ubuntu 20.04 is a distribution that we can feasibly use for meaningful 2024.1 testing. This implies, in my opinion, that there is a solid reason for dropping python 3.8 support in advance of it going EOL, as required by [0]. Looking at the Python Update Process resolution [1], python 3.8 does not meet the three criteria set out in the "Unit Tests" section: 1. it's not the latest version of Python 3 available in any distro we can feasibly use for testing 2. It's not the default in any of the Linux distros identified in the 2024.1 PTI 3. It isn't used to run integration tests at the beginning of the 2024.1 (Caracal) cycle Add to that the fact that libraries are beginning to drop support [2], add further that py38 will go EOL roughly 6 months after the 2024.1 release (no more security updates), I don't see a reason to wait until a key library forces us to make a change during the development cycle. I'd prefer to do it now. [0] https://governance.openstack.org/tc/reference/pti/python.html#specific-comma... [1] https://governance.openstack.org/tc/resolutions/20181024-python-update-proce... [2] https://review.opendev.org/c/openstack/requirements/+/884564
To be fair, all these arguments were also raised during the 2023.2 development cycle. With additional one - we will be not able to drop support during non-SLURP release without breaking upgrades/compatibility between SLURPs. A decision was taken to keep py3.8 back then with acknowledgement of that fact. Now, when 2023.2 has been released with py3.8 support, from my perspective there is really no way it can be removed until 2024.1 without breaking someone's upgrades. Despite I agree with your arguments, TC has quite explicitly stated that deprecations should not take place in non-SLURP releases according to [1] and I believe that reasons and background in this resolution are still valid and should be respected, despite how inconvenient it might be sometimes. As also platform consumers rely on that and plan their maintenance and further work based on promises that are being made by community. In case a key library drops support for py3.8 then I believe we will need to leverage upper-constraints to set the version back for the library to the one which supports py3.8. Either for all python versions, or solely for py3.8, like we already did for py3.6 and py3.8 back in Xena/Yoga. [1] https://governance.openstack.org/tc/resolutions/20220210-release-cadence-adj... On Wed, Oct 11, 2023, 15:45 Brian Rosmaita <rosmaita.fossdev@gmail.com> wrote:
I thought about this issue some more after yesterday's TC meeting, and tried to articulate why I think keeping python 3.8 support for 2024.1 is a bad idea on the patch:
https://review.opendev.org/c/openstack/governance/+/895160
I don't know if my argument there will change anyone's mind, but since a lot of people have already voted on the gerrit review, I wanted to make sure that people are at least aware of it, to wit:
I just don't see that py38 support for 2024.1 makes sense. It's not a default version in Ubuntu 22.04, Debian 12, Debian 11, CentOS Stream 9, or Rocky Linux 9, which are the distros specifically called out in the current 2024.1 PTI that this proposal is patching.
While we can use Ubuntu 20.04 to run unit tests, we can't run master (2024.1 development) devstack in it, so it doesn't seem to me that Ubuntu 20.04 is a distribution that we can feasibly use for meaningful 2024.1 testing. This implies, in my opinion, that there is a solid reason for dropping python 3.8 support in advance of it going EOL, as required by [0].
Looking at the Python Update Process resolution [1], python 3.8 does not meet the three criteria set out in the "Unit Tests" section:
1. it's not the latest version of Python 3 available in any distro we can feasibly use for testing 2. It's not the default in any of the Linux distros identified in the 2024.1 PTI 3. It isn't used to run integration tests at the beginning of the 2024.1 (Caracal) cycle
Add to that the fact that libraries are beginning to drop support [2], add further that py38 will go EOL roughly 6 months after the 2024.1 release (no more security updates), I don't see a reason to wait until a key library forces us to make a change during the development cycle. I'd prefer to do it now.
[0]
https://governance.openstack.org/tc/reference/pti/python.html#specific-comma... [1]
https://governance.openstack.org/tc/resolutions/20181024-python-update-proce... [2] https://review.opendev.org/c/openstack/requirements/+/884564
On Wed, Oct 11, 2023, at 10:19 AM, Dmitriy Rabotyagov wrote:
To be fair, all these arguments were also raised during the 2023.2 development cycle. With additional one - we will be not able to drop support during non-SLURP release without breaking upgrades/compatibility between SLURPs. A decision was taken to keep py3.8 back then with acknowledgement of that fact.
Now, when 2023.2 has been released with py3.8 support, from my perspective there is really no way it can be removed until 2024.1 without breaking someone's upgrades.
I'm not sure this is the case? 2023.1 and 2023.2 (the two releases you might upgrade from to 2024.1) both support python 3.9 and python3.10 in addition to python3.8. Your upgrade path would be to first update your runtime on 2023.x from python3.8 to 3.9/3.10 then upgrade to 2024.1.
Despite I agree with your arguments, TC has quite explicitly stated that deprecations should not take place in non-SLURP releases according to [1] and I believe that reasons and background in this resolution are still valid and should be respected, despite how inconvenient it might be sometimes. As also platform consumers rely on that and plan their maintenance and further work based on promises that are being made by community.
In case a key library drops support for py3.8 then I believe we will need to leverage upper-constraints to set the version back for the library to the one which supports py3.8. Either for all python versions, or solely for py3.8, like we already did for py3.6 and py3.8 back in Xena/Yoga.
I think the big struggle with dropping python3.8 previously was the order of operates was backwards. Services need to drop python versions first, then libraries. This will require coordination across openstack which is what we didn't have the last time we attempted this.
[1] https://governance.openstack.org/tc/resolutions/20220210-release-cadence-adj...
---- On Wed, 11 Oct 2023 10:31:54 -0700 Clark Boylan wrote ---
On Wed, Oct 11, 2023, at 10:19 AM, Dmitriy Rabotyagov wrote:
To be fair, all these arguments were also raised during the 2023.2 development cycle. With additional one - we will be not able to drop support during non-SLURP release without breaking upgrades/compatibility between SLURPs. A decision was taken to keep py3.8 back then with acknowledgement of that fact.
Now, when 2023.2 has been released with py3.8 support, from my perspective there is really no way it can be removed until 2024.1 without breaking someone's upgrades.
I'm not sure this is the case? 2023.1 and 2023.2 (the two releases you might upgrade from to 2024.1) both support python 3.9 and python3.10 in addition to python3.8. Your upgrade path would be to first update your runtime on 2023.x from python3.8 to 3.9/3.10 then upgrade to 2024.1.
Despite I agree with your arguments, TC has quite explicitly stated that deprecations should not take place in non-SLURP releases according to [1] and I believe that reasons and background in this resolution are still valid and should be respected, despite how inconvenient it might be sometimes. As also platform consumers rely on that and plan their maintenance and further work based on promises that are being made by community.
In case a key library drops support for py3.8 then I believe we will need to leverage upper-constraints to set the version back for the library to the one which supports py3.8. Either for all python versions, or solely for py3.8, like we already did for py3.6 and py3.8 back in Xena/Yoga.
I think the big struggle with dropping python3.8 previously was the order of operates was backwards. Services need to drop python versions first, then libraries. This will require coordination across openstack which is what we didn't have the last time we attempted this.
Yes, and that is why I think this work need more coordination and community-wide goal is one way to do so like we do for any distro version upgrade in our CI. -gmann
[1] https://governance.openstack.org/tc/resolutions/20220210-release-cadence-adj...
To be fair, all these arguments were also raised during the 2023.2 development cycle. With additional one - we will be not able to drop support during non-SLURP release without breaking upgrades/compatibility between SLURPs. A decision was taken to keep py3.8 back then with acknowledgement of that fact.
Now, when 2023.2 has been released with py3.8 support, from my perspective there is really no way it can be removed until 2024.1 without breaking someone's upgrades.
Despite I agree with your arguments, TC has quite explicitly stated that deprecations should not take place in non-SLURP releases according to [1] deprecateion are allowed in non slurps we just cant remove supprot for a depreaction
On Wed, 2023-10-11 at 19:19 +0200, Dmitriy Rabotyagov wrote: that has not been release in slurp. ingoring that for a moment the deprecation poilcy only applies to aplciation feature we have never applied it to runtimes or minium python version.
and I believe that reasons and background in this resolution are still valid and should be respected, despite how inconvenient it might be sometimes. As also platform consumers rely on that and plan their maintenance and further work based on promises that are being made by community.
In case a key library drops support for py3.8 then I believe we will need to leverage upper-constraints to set the version back for the library to the one which supports py3.8. Either for all python versions, or solely for py3.8, like we already did for py3.6 and py3.8 back in Xena/Yoga.
[1] https://governance.openstack.org/tc/resolutions/20220210-release-cadence-adj...
On Wed, Oct 11, 2023, 15:45 Brian Rosmaita <rosmaita.fossdev@gmail.com> wrote:
I thought about this issue some more after yesterday's TC meeting, and tried to articulate why I think keeping python 3.8 support for 2024.1 is a bad idea on the patch:
https://review.opendev.org/c/openstack/governance/+/895160
I don't know if my argument there will change anyone's mind, but since a lot of people have already voted on the gerrit review, I wanted to make sure that people are at least aware of it, to wit:
I just don't see that py38 support for 2024.1 makes sense. It's not a default version in Ubuntu 22.04, Debian 12, Debian 11, CentOS Stream 9, or Rocky Linux 9, which are the distros specifically called out in the current 2024.1 PTI that this proposal is patching.
While we can use Ubuntu 20.04 to run unit tests, we can't run master (2024.1 development) devstack in it, so it doesn't seem to me that Ubuntu 20.04 is a distribution that we can feasibly use for meaningful 2024.1 testing. This implies, in my opinion, that there is a solid reason for dropping python 3.8 support in advance of it going EOL, as required by [0].
Looking at the Python Update Process resolution [1], python 3.8 does not meet the three criteria set out in the "Unit Tests" section:
1. it's not the latest version of Python 3 available in any distro we can feasibly use for testing 2. It's not the default in any of the Linux distros identified in the 2024.1 PTI 3. It isn't used to run integration tests at the beginning of the 2024.1 (Caracal) cycle
Add to that the fact that libraries are beginning to drop support [2], add further that py38 will go EOL roughly 6 months after the 2024.1 release (no more security updates), I don't see a reason to wait until a key library forces us to make a change during the development cycle. I'd prefer to do it now.
[0]
https://governance.openstack.org/tc/reference/pti/python.html#specific-comma... [1]
https://governance.openstack.org/tc/resolutions/20181024-python-update-proce... [2] https://review.opendev.org/c/openstack/requirements/+/884564
Your upgrade path would be to first update your runtime on 2023.x from
deprecateion are allowed in non slurps we just cant remove supprot for a depreaction that has not been release in slurp. Yes, thanks for correcting me, that's eventually what I meant, though picked misguiding words to the express that. ingoring that for a moment the deprecation poilcy only applies to aplciation feature we have never applied it to runtimes or minium python version. I'm not sure it's specified in resolution? To be frank, dropping an application feature is way less deal breaker then dropping python version, imo. I would compare it more with bumping min libvirt version then (which is not supported by some platform). python3.8 to 3.9/3.10 then upgrade to 2024.1. Sorry, I can't agree here with you Clark. With that we can also drop platforms as well and tell users to upgrade OS in between of SLURP releases. I know I m a bit exaggerating here, but it's kinda close. Simple example of operations life - we have to schedule all planned maintenances in a 6month before doing them. And if our plan was, for example, involving openstack upgrade to 2024.1, but then, from the blue sky, despite decision that was just taken by TC, I see that I need to do some OS upgrades before that, it would break plans quite dramatically. This would put into position, that by the time of the next possible planned maintenance OpenStack version I am using (and was supposed to be upgraded 6 month ago) is already unmaintained. I am not saying that py3.8 drop is affecting us specifically in this way (dropping a platform would though), but just giving a perspective that such decisions might result in quite some overhead for end users, especially when they in a way contradict with existing TC decisions. I'm not even saying that such thing can easily deduct available time for upstream contributions... But also there should be a trust in TC decisions from end users, so that they know these can't change overnight and can be relied on. As otherwise it would be pretty much hard to convince enyone, that you can rely on OpenStack as a project.
On Wed, Oct 11, 2023, at 1:41 PM, Dmitriy Rabotyagov wrote:
deprecateion are allowed in non slurps we just cant remove supprot for a depreaction that has not been release in slurp.
Yes, thanks for correcting me, that's eventually what I meant, though picked misguiding words to the express that.
ingoring that for a moment the deprecation poilcy only applies to aplciation feature we have never applied it to runtimes or minium python version.
I'm not sure it's specified in resolution? To be frank, dropping an application feature is way less deal breaker then dropping python version, imo. I would compare it more with bumping min libvirt version then (which is not supported by some platform).
Your upgrade path would be to first update your runtime on 2023.x from python3.8 to 3.9/3.10 then upgrade to 2024.1.
Sorry, I can't agree here with you Clark. With that we can also drop platforms as well and tell users to upgrade OS in between of SLURP releases.
I know I m a bit exaggerating here, but it's kinda close.
I think users needing to catch up on external dependencies at the boundaries between slurp upgrades is the expectation, and what I described takes that into account. Basically for a release X consider any releases A, B, C that a user might be able to directly upgrade to X from. Make that possible. In this case we have through overlapping python versions on A and B between old and new allowing you to get to new on X. It is more than an exaggeration: it just isn't realistic to support anything else in my opinion. You'll be stuck supporting ancient everything if you don't take a position like this.
Simple example of operations life - we have to schedule all planned maintenances in a 6month before doing them. And if our plan was, for example, involving openstack upgrade to 2024.1, but then, from the blue sky, despite decision that was just taken by TC, I see that I need to do some OS upgrades before that, it would break plans quite dramatically. This would put into position, that by the time of the next possible planned maintenance OpenStack version I am using (and was supposed to be upgraded 6 month ago) is already unmaintained.
I am not saying that py3.8 drop is affecting us specifically in this way (dropping a platform would though), but just giving a perspective that such decisions might result in quite some overhead for end users, especially when they in a way contradict with existing TC decisions. I'm not even saying that such thing can easily deduct available time for upstream contributions...
But also there should be a trust in TC decisions from end users, so that they know these can't change overnight and can be relied on. As otherwise it would be pretty much hard to convince enyone, that you can rely on OpenStack as a project.
We are one week post 2023.2 release. I think now is exactly the time to figure this stuff out for 2024.1. Yes, it could be done earlier, but I don't think it is too late to make decisions that allow the software to be sustainable into the future.
It is more than an exaggeration: it just isn't realistic to support anything else in my opinion. You'll be stuck supporting ancient everything if you don't take a position like this.
Then I'd say that SLURP approach should be cancelled as I'm not sure any more what real-users situation it is solving then. And if 1 year is a too long period which makes everything "ancient". Because as I said, removal of py3.8 had been raised at beginning of 2023.2 (and I bet everyone recall the mess we got into because of that), when we _really_ could drop it, but just by handling that in a better way. But now in non-SLURP I feel pretty much wrong in doing so. And again, I am not protecting py3.8 specifically, but I am standing for expectations that we set by introducing SLURPs, which IMO was one of the biggest deals for operators lately.
On Wed, Oct 11, 2023, at 2:05 PM, Dmitriy Rabotyagov wrote:
It is more than an exaggeration: it just isn't realistic to support anything else in my opinion. You'll be stuck supporting ancient everything if you don't take a position like this.
Then I'd say that SLURP approach should be cancelled as I'm not sure any more what real-users situation it is solving then. And if 1 year is a too long period which makes everything "ancient".
It isn't one year. Python 3.8 has been out since October 2019, was part of the Ubuntu Focal release in April 2020, was first supported in OpenStack Victoria (October 2020), and if kept as part of the 2024.1 release would be kept alive in OpenStack until approximately October 2025. Python 3.8 will be EOL'd by its maintainers in ~October 2024. We have seen that when python releases become EOL'd many libraries stop supporting that release. This potentially leads to gaps in security and general bugfixing. I think giving users a clear signal of when they should update runtimes before that runtime becomes problematic is a good thing.
Because as I said, removal of py3.8 had been raised at beginning of 2023.2 (and I bet everyone recall the mess we got into because of that), when we _really_ could drop it, but just by handling that in a better way. But now in non-SLURP I feel pretty much wrong in doing so.
This is the part I do not understand. What is the functional difference for operators going through an upgrade process if we drop it in the .1 release vs the .2 release? Let's draw up the resulting upgrade paths. If 2023.2 had dropped python3.8: * Run 2023.1 on python3.8 * Convert 2023.1 deployment to python3.10 * Upgrade 2023.1 to 2023.2 or 2024.1 If 2024.1 drops python3.8 support: * Run 2023.1 on python3.8 * Optionally upgrade deployment to 2023.2 * Convert 2023.x to python3.10 * Upgrade to 2024.1 There isn't a huge difference here. And in both cases users can still skip an OpenStack version in the upgrade process if they choose to do so. The only real difference is if we want people to continue running 2024.1 on python3.8 before converting to probably python3.11 or python3.12 at that point. I don't think that is the sort of signal we want to give because deployments in that situation are very likely to run into struggles with library updates.
And again, I am not protecting py3.8 specifically, but I am standing for expectations that we set by introducing SLURPs, which IMO was one of the biggest deals for operators lately.
Hi, Am 12. Oktober 2023 01:23:16 MESZ schrieb Clark Boylan <cboylan@sapwetik.org>:
It isn't one year. Python 3.8 has been out since October 2019, was part of the Ubuntu Focal release in April 2020, was first supported in OpenStack Victoria (October 2020), and if kept as part of the 2024.1 release would be kept alive in OpenStack until approximately October 2025.
Python 3.8 will be EOL'd by its maintainers in ~October 2024. We have seen that when python releases become EOL'd many libraries stop supporting that release. This potentially leads to gaps in security and general bugfixing. I think giving users a clear signal of when they should update runtimes before that runtime becomes problematic is a good thing.
Fully agree. A strong deprecation is definitely appropriate as 2024.1 on py3.8 can only be reliably supported with maintenance for ~1/2 year. A warning message whenever you try to run 2024.1 code on py3.8. Leaves the question open whether or not we want to invest (likely small) effort to keep py3. 8 technically working or not. Do we consider treating clients different from server side? I.e. avoid supporting running OpenStack 2024.1 on py3.8 while being more relaxed with users that want to run sdk & client-tools on py3.8 (Ubuntu 20.04LTS)? PS: I don't see a strong relationship with SLURP or non-SLURP. Upgrades always go through a number of steps and the important thing is that we take responsibility for documenting and validating them. HTH, -- Kurt
Hi, Dnia czwartek, 12 października 2023 01:23:16 CEST Clark Boylan pisze:
On Wed, Oct 11, 2023, at 2:05 PM, Dmitriy Rabotyagov wrote:
It is more than an exaggeration: it just isn't realistic to support anything else in my opinion. You'll be stuck supporting ancient everything if you don't take a position like this.
Then I'd say that SLURP approach should be cancelled as I'm not sure any more what real-users situation it is solving then. And if 1 year is a too long period which makes everything "ancient".
It isn't one year. Python 3.8 has been out since October 2019, was part of the Ubuntu Focal release in April 2020, was first supported in OpenStack Victoria (October 2020), and if kept as part of the 2024.1 release would be kept alive in OpenStack until approximately October 2025.
Python 3.8 will be EOL'd by its maintainers in ~October 2024. We have seen that when python releases become EOL'd many libraries stop supporting that release. This potentially leads to gaps in security and general bugfixing. I think giving users a clear signal of when they should update runtimes before that runtime becomes problematic is a good thing.
Because as I said, removal of py3.8 had been raised at beginning of 2023.2 (and I bet everyone recall the mess we got into because of that), when we _really_ could drop it, but just by handling that in a better way. But now in non-SLURP I feel pretty much wrong in doing so.
This is the part I do not understand. What is the functional difference for operators going through an upgrade process if we drop it in the .1 release vs the .2 release? Let's draw up the resulting upgrade paths.
If 2023.2 had dropped python3.8:
* Run 2023.1 on python3.8 * Convert 2023.1 deployment to python3.10 * Upgrade 2023.1 to 2023.2 or 2024.1
If 2024.1 drops python3.8 support:
* Run 2023.1 on python3.8 * Optionally upgrade deployment to 2023.2 * Convert 2023.x to python3.10 * Upgrade to 2024.1
There isn't a huge difference here. And in both cases users can still skip an OpenStack version in the upgrade process if they choose to do so. The only real difference is if we want people to continue running 2024.1 on python3.8 before converting to probably python3.11 or python3.12 at that point. I don't think that is the sort of signal we want to give because deployments in that situation are very likely to run into struggles with library updates.
That's exactly my understanding. I don't see big difference there too.
And again, I am not protecting py3.8 specifically, but I am standing for expectations that we set by introducing SLURPs, which IMO was one of the biggest deals for operators lately.
-- Slawek Kaplonski Principal Software Engineer Red Hat
It isn't one year.
We could drop things in April 2023 (for 2023.2, which we didn't do), or we can drop them in April 2024 (for 2024.2), which is roughly 1 year? I think I was talking more about ability to strategically plan such removal so that they were not matching non-SLURP releases.
Python 3.8 will be EOL'd by its maintainers in ~October 2024. We have seen that when python releases become EOL'd many libraries stop supporting that release. This potentially leads to gaps in security and general bugfixing.
Well, while I do agree with concerns here, I think it's missing some important notes: 1. Py3.8. is going to be supported by Ubuntu 20.04 until April 2025 at worst, and then there's also ESM till 2030. I bet RHEL has smth like that as well. So security backports for python packages and python itself is not really our pain. 2. We still can test things quite consistently, since we have all our dependencies maintained in upper-constraints.. Basically due to U-C I can build and install Ocata as of today with Py2.7 (I'm not saying it's good or bad, just stating the fact). Also the proposal is quite clear to minimize testing to bare minimal, i.e. unit tests, so I don't see how it's a deal breaker here. And it's in fact an operator's responsibility to ensure their environments are secure enough.
If 2023.2 had dropped python3.8:
* Run 2023.1 on python3.8 * Convert 2023.1 deployment to python3.10 * Upgrade 2023.1 to 2023.2 or 2024.1
Yes, that is right. Important thing here is that it would be expected. Like I know in advance (with a year in my pocket), that in order to upgrade to 2024.1 or to 2023.2 I need to upgrade python first, and that is totally fine. So such thing can be planned, tested and executed in 2 separate steps.
If 2024.1 drops python3.8 support:
* Run 2023.1 on python3.8 * Optionally upgrade deployment to 2023.2 * Convert 2023.x to python3.10 * Upgrade to 2024.1
And here it's not precisely correct. As with that, upgrade to 2023.2 is not _really_ optional. And such upgrade (having temporary version in between) from operational perspective is really a nightmare: * Firstly it would take a 3x more time get the upgrade done (as you're in fact doing 3 upgrades) * Secondly you need to somehow explain the users that the API versions (or microversions) you get in between is not smth you should get used to, because it they will really soon * Should you also upload new images for magnum/amphora/trove and failover everything for this intermediate upgrade? * Way more time you need to spend on testing such upgrade, as there's another release now that needs to be handled. * Last, but not least, each upgrade still brings some disturbance (like restart of heat-engine might got stack borked), and now you need to do that twice So eventually, with all that, you come to the same flow as you described if 2023.2 had dropped python3.8, meaning you should upgrade your 2023.1 deployment to python3.10 and then proceed to 2024.1. Except: 1. You relied on TC's decision that things will not be removed on non-SLURP so invested your time into 2024.1 upgrade preparation(and maybe development of features for 2024.1). Now you need to scrap all your upgrade (or contribution) plans and plan python upgrade internally instead, as this becomes basically a pre-requirement. 2. Python upgrade for operations is not only about OpenStack, actually. As you also need to adopt your tooling, scripts and plenty of stuff to work against the new Python version. 3. If you've already notified end-users to expect new features that 2024.1 will bring to them in 6 month - you have to disappoint them now, since upgrades won't happen in that timeline, because you need to deal with other things.
I don't see a strong relationship with SLURP or non-SLURP
In the motivation part for SLURP there's that thing: "Upgrades will be supported between “SLURP” releases, in addition to between adjacent major releases (as they are today)." IMO, with removing py38 now we're breaking that. If you're on 2023.1 and running py3.8 you can't just jump to 2024.1, which means that this upgrade path is not _really_ supported. Or we really need to clarify what we mean under "supported upgrades". I see how I can be wrong in that assumption and that there're arguments against this vision. But I assume this perspective has a right to exist as well. Again. I really don't care about py3.8 as we are not running it since upgrade to 2023.1 as switched to py3.10 right after it. I'm more fighting for consistency between our decisions and actions that we take afterwards, so that any openstack user or operator could rely on decisions that were taken and promoted, so that OpenStack was perceived as a solid platform. чт, 12 окт. 2023 г. в 09:42, Sławek Kapłoński <skaplons@redhat.com>:
Hi,
Dnia czwartek, 12 października 2023 01:23:16 CEST Clark Boylan pisze:
On Wed, Oct 11, 2023, at 2:05 PM, Dmitriy Rabotyagov wrote:
It is more than an exaggeration: it just isn't realistic to support anything else in my opinion. You'll be stuck supporting ancient everything if you don't take a position like this.
Then I'd say that SLURP approach should be cancelled as I'm not sure any more what real-users situation it is solving then. And if 1 year is a too long period which makes everything "ancient".
It isn't one year. Python 3.8 has been out since October 2019, was part of the Ubuntu Focal release in April 2020, was first supported in OpenStack Victoria (October 2020), and if kept as part of the 2024.1 release would be kept alive in OpenStack until approximately October 2025.
Python 3.8 will be EOL'd by its maintainers in ~October 2024. We have seen that when python releases become EOL'd many libraries stop supporting that release. This potentially leads to gaps in security and general bugfixing. I think giving users a clear signal of when they should update runtimes before that runtime becomes problematic is a good thing.
Because as I said, removal of py3.8 had been raised at beginning of 2023.2 (and I bet everyone recall the mess we got into because of that), when we _really_ could drop it, but just by handling that in a better way. But now in non-SLURP I feel pretty much wrong in doing so.
This is the part I do not understand. What is the functional difference for operators going through an upgrade process if we drop it in the .1 release vs the .2 release? Let's draw up the resulting upgrade paths.
If 2023.2 had dropped python3.8:
* Run 2023.1 on python3.8 * Convert 2023.1 deployment to python3.10 * Upgrade 2023.1 to 2023.2 or 2024.1
If 2024.1 drops python3.8 support:
* Run 2023.1 on python3.8 * Optionally upgrade deployment to 2023.2 * Convert 2023.x to python3.10 * Upgrade to 2024.1
There isn't a huge difference here. And in both cases users can still skip an OpenStack version in the upgrade process if they choose to do so. The only real difference is if we want people to continue running 2024.1 on python3.8 before converting to probably python3.11 or python3.12 at that point. I don't think that is the sort of signal we want to give because deployments in that situation are very likely to run into struggles with library updates.
That's exactly my understanding. I don't see big difference there too.
And again, I am not protecting py3.8 specifically, but I am standing for expectations that we set by introducing SLURPs, which IMO was one of the biggest deals for operators lately.
-- Slawek Kaplonski Principal Software Engineer Red Hat
On Thu, 2023-10-12 at 11:35 +0200, Dmitriy Rabotyagov wrote:
It isn't one year.
We could drop things in April 2023 (for 2023.2, which we didn't do), actully we did.
we drop supprot for py38 and hten re-added it because some project were not ready to drop it. i stongly belive that was a mistake. to be clear the 2023.1 release was intened to be the final release to suport python 3.8 and ubuntu 20.04 https://github.com/openstack/governance/blob/master/reference/runtimes/2023.... it was the release where ubuntu 20.04 was supproted for one addtional release for smooth upgrades. when the 2023.2 runtime was selected we removed support for ubuntu 20.04 because all user of 2023.2 are expected to have upgraded to ubuntu 22.04 prior to upgrading openstack to 2023.2 https://github.com/openstack/governance/commit/6ecb9c7210b152bc4ad36d1e830f4... is the orgianl 2023.2 testing runtime and then in may python 3.8 was readded https://github.com/openstack/governance/commit/05ac02c2acd6117092c628fb5b04a... i really dont think that the correct decsion espically after we had already started removing support for py38
or we can drop them in April 2024 (for 2024.2), which is roughly 1 year? I think I was talking more about ability to strategically plan such removal so that they were not matching non-SLURP releases. if we dont drop supprot in 2024.1 then i think we must drop it in 2024.2 i dont think it would be reasonable to ask porject to continue to supprot it and i honestly dont think it is a reasonbale ask for 2024.1 or 2023.2 i was very unhappy with teh late readdtion of it in may after it was finally removed.
Python 3.8 will be EOL'd by its maintainers in ~October 2024. We have seen that when python releases become EOL'd many libraries stop supporting that release. This potentially leads to gaps in security and general bugfixing.
Well, while I do agree with concerns here, I think it's missing some important notes: 1. Py3.8. is going to be supported by Ubuntu 20.04 until April 2025 at worst, and then there's also ESM till 2030. I bet RHEL has smth like that as well. So security backports for python packages and python itself is not really our pain. we do not use packages form the disto in our testing outside the standar libary and intepreter so it is a problem for the libs we use from pypi 2. We still can test things quite consistently, since we have all our dependencies maintained in upper-constraints.. Basically due to U-C I can build and install Ocata as of today with Py2.7 (I'm not saying it's good or bad, just stating the fact) not entirly. in ci or a contienr yes but some of our deps will not compile on newer operating systems. i.e. to run nova's pep8 tox env on wallaby i need to create an vm/container to do that because it wont actully work on a modern operating system. the system packages that we leverage are too new and cause issues.
. Also the proposal is quite clear to minimize testing to bare minimal, i.e. unit tests, so I don't see how it's a deal breaker here. And it's in fact an operator's responsibility to ensure their environments are secure enough.
If 2023.2 had dropped python3.8:
* Run 2023.1 on python3.8 * Convert 2023.1 deployment to python3.10 * Upgrade 2023.1 to 2023.2 or 2024.1
Yes, that is right. Important thing here is that it would be expected. Like I know in advance (with a year in my pocket), that in order to upgrade to 2024.1 or to 2023.2 I need to upgrade python first, and that is totally fine. So such thing can be planned, tested and executed in 2 separate steps. you have already been given that notice as we notified everyone that ubuntu 20.04 was last supported in 2023.1 an all othe supported distos in that release ship python 3.9+ debian 11 and centos 9 stream both already use python 3.9, ubuntu 22.04 ships 3.10
so this is not a issue for the removal of python 3.8 i dont know of any suppproted distribution where openstack 2023.2+ was shiped on py38 so keeping supprot in 2023.2 or 2024.1 seam academic to me. do we know of any operator or deployment that would actully deploy this way?
If 2024.1 drops python3.8 support:
* Run 2023.1 on python3.8 * Optionally upgrade deployment to 2023.2 * Convert 2023.x to python3.10 * Upgrade to 2024.1
this was not intended to be a supproted upgrade path on yoga and zed the only supported runtime that had python 3.8 was ubuntu 20.04 https://github.com/openstack/governance/blob/master/reference/runtimes/yoga.... https://github.com/openstack/governance/blob/master/reference/runtimes/zed.r... so when you upgraded to 2023.1 ( the first slurp release) you did that on 20.04 then for smooth upgrade we supported ubuntu 20.04 and 22.04 in antelop(2023.1) so the expecation was if your on ubuntu using 20.04 and 2023.1 you would upgrade to 22.04 before doing any futher openstack upgrades. ill note that canonical has supported openstack on ubuntu 22.04 since yoga https://ubuntu.com/openstack/docs/supported-versions so they expected people to upgrade form 20.04 to 22.04 after upgrading to yoga on 18.04. we dropped supprot for it well after when any operator should have still be using it if the installed openstack form packages. what is also relevant to this conversation is openstack 2024.1 will be supported on ubuntu 22.04 and 24.04 by canonical for smoth upgrade and we will additionally suprot 2024.2 on 22.04 for smoth upgrade. so to go form 2023.1 -> 2023.2 or 2024.1 the upgrade path is oepnstack 2023.1 to oepnstack 2023.1 on ubuntu 22.04 and python 3.10 then and only then can you upgrade to 2023.2 or 2024.1 note that nova does not support the libvirt/qemu shiped on ubuntu 20.04 in 2023.2 we declared our next min version in wallaby and deferd the bump twice as it was orgianlly planned for z so all users of the the 2023.1 slrup release were notified that this was going to happen well before we did it.
And here it's not precisely correct. As with that, upgrade to 2023.2 is not _really_ optional.
And such upgrade (having temporary version in between) from operational perspective is really a nightmare: * Firstly it would take a 3x more time get the upgrade done (as you're in fact doing 3 upgrades) * Secondly you need to somehow explain the users that the API versions (or microversions) you get in between is not smth you should get used to, because it they will really soon * Should you also upload new images for magnum/amphora/trove and failover everything for this intermediate upgrade? * Way more time you need to spend on testing such upgrade, as there's another release now that needs to be handled. * Last, but not least, each upgrade still brings some disturbance (like restart of heat-engine might got stack borked), and now you need to do that twice
So eventually, with all that, you come to the same flow as you described if 2023.2 had dropped python3.8, meaning you should upgrade your 2023.1 deployment to python3.10 and then proceed to 2024.1. yes that is the expected flow. upgrade to 3.9+ on 2023.1 before proceeding to 2024.1 Except: 1. You relied on TC's decision that things will not be removed on non-SLURP so invested your time into 2024.1 upgrade preparation(and maybe development of features for 2024.1). Now you need to scrap all your upgrade (or contribution) plans and plan python upgrade internally instead, as this becomes basically a pre-requirement. i would consieer anyoen that takes that perspecte to have miss understood
2. Python upgrade for operations is not only about OpenStack, actually. As you also need to adopt your tooling, scripts and plenty of stuff to work against the new Python version. 3. If you've already notified end-users to expect new features that 2024.1 will bring to them in 6 month - you have to disappoint them now, since upgrades won't happen in that timeline, because you need to deal with other things.
it is optional ubuntu 20.04 and python 3.8 was not intended to be supprot in 2023.2 which is why you are ment to do the os upgrade and python upgrade on 2023.1 or eariler. the intent of our supproted runtimes and tc decisions. im sorry but form my perspective we did not agree to supproting python 3.8 until 2024.1 when we were selecting the the runtime for 2023.1 or agreeing on the new slurp process. form my perspective it was clear to me that 2023.1 was intened to be the last release to supprot eith ubuntu 20.04 or python 3.8 form my perspecitve we notified them that 2023.1 would be the last release to supprot python 3.8 and then revers couse on that in 2023.2 for what i condider to be invalid reasons. yes i know our ci broke when we start enforce a min version but i think we shoudl have fixed the project that coudl not drop 38 not readded it.
I don't see a strong relationship with SLURP or non-SLURP
In the motivation part for SLURP there's that thing: "Upgrades will be supported between “SLURP” releases, in addition to between adjacent major releases (as they are today)." IMO, with removing py38 now we're breaking that.
and in my view py38 was deprecated for removal in 2023.1 so there is no breakage.
If you're on 2023.1 and running py3.8 you can't just jump to 2024.1, which means that this upgrade path is not _really_ supported. Or we really need to clarify what we mean under "supported upgrades". I see how I can be wrong in that assumption and that there're arguments against this vision. But I assume this perspective has a right to exist as well.
Again. I really don't care about py3.8 as we are not running it since upgrade to 2023.1 as switched to py3.10 right after it. I'm more fighting for consistency between our decisions and actions that we take afterwards, so that any openstack user or operator could rely on decisions that were taken and promoted, so that OpenStack was perceived as a solid platform. i think retoactivly adding platform and python version that were previously agreed to be drop undermines that platfrom and our ablity to maintain it. to be clear as a contibutoer and core review i felt underminded by the tc when python 3.8 was readded in may. and i felt like we sent the wrong message to operators because we said somethign was supproted when none of the testing runtim operationg systems supproted a unifed deployment of openstack on that versoin of python.
чт, 12 окт. 2023 г. в 09:42, Sławek Kapłoński <skaplons@redhat.com>:
Hi,
Dnia czwartek, 12 października 2023 01:23:16 CEST Clark Boylan pisze:
On Wed, Oct 11, 2023, at 2:05 PM, Dmitriy Rabotyagov wrote:
It is more than an exaggeration: it just isn't realistic to support anything else in my opinion. You'll be stuck supporting ancient everything if you don't take a position like this.
Then I'd say that SLURP approach should be cancelled as I'm not sure any more what real-users situation it is solving then. And if 1 year is a too long period which makes everything "ancient".
It isn't one year. Python 3.8 has been out since October 2019, was part of the Ubuntu Focal release in April 2020, was first supported in OpenStack Victoria (October 2020), and if kept as part of the 2024.1 release would be kept alive in OpenStack until approximately October 2025.
Python 3.8 will be EOL'd by its maintainers in ~October 2024. We have seen that when python releases become EOL'd many libraries stop supporting that release. This potentially leads to gaps in security and general bugfixing. I think giving users a clear signal of when they should update runtimes before that runtime becomes problematic is a good thing.
Because as I said, removal of py3.8 had been raised at beginning of 2023.2 (and I bet everyone recall the mess we got into because of that), when we _really_ could drop it, but just by handling that in a better way. But now in non-SLURP I feel pretty much wrong in doing so.
This is the part I do not understand. What is the functional difference for operators going through an upgrade process if we drop it in the .1 release vs the .2 release? Let's draw up the resulting upgrade paths.
If 2023.2 had dropped python3.8:
* Run 2023.1 on python3.8 * Convert 2023.1 deployment to python3.10 * Upgrade 2023.1 to 2023.2 or 2024.1
If 2024.1 drops python3.8 support:
* Run 2023.1 on python3.8 * Optionally upgrade deployment to 2023.2 * Convert 2023.x to python3.10 * Upgrade to 2024.1
There isn't a huge difference here. And in both cases users can still skip an OpenStack version in the upgrade process if they choose to do so. The only real difference is if we want people to continue running 2024.1 on python3.8 before converting to probably python3.11 or python3.12 at that point. I don't think that is the sort of signal we want to give because deployments in that situation are very likely to run into struggles with library updates.
That's exactly my understanding. I don't see big difference there too.
And again, I am not protecting py3.8 specifically, but I am standing for expectations that we set by introducing SLURPs, which IMO was one of the biggest deals for operators lately.
-- Slawek Kaplonski Principal Software Engineer Red Hat
we drop supprot for py38 and hten re-added it because some project were not ready to drop it. i stongly belive that was a mistake.
Well, I'm not ready to judge if it was a mistake or not - there were good arguments from both sides. But from projects perspective (not libraries) it was pretty much in line to stop supporting it for 2023.2, yes.
if we dont drop supprot in 2024.1 then i think we must drop it in 2024.2 i dont think it would be reasonable to ask porject to continue to supprot it and i honestly dont think it is a reasonbale ask for 2024.1 or 2023.2 i was very unhappy with teh late readdtion of it in may after it was finally removed.
Yes, absolutely, it must go away in 2024.2 for sure.
1. Py3.8. is going to be supported by Ubuntu 20.04 until April 2025 at worst, and then there's also ESM till 2030. I bet RHEL has smth like that as well. So security backports for python packages and python itself is not really our pain. we do not use packages form the disto in our testing outside the standar libary and intepreter so it is a problem for the libs we use from pypi
But how much is the security of packages a concern inside a containerized CI, when we're talking about running unit testing only (and not dropping it from setup.cfg)?
2. We still can test things quite consistently, since we have all our dependencies maintained in upper-constraints.. Basically due to U-C I can build and install Ocata as of today with Py2.7 (I'm not saying it's good or bad, just stating the fact) not entirly. in ci or a contienr yes but some of our deps will not compile on newer operating systems. i.e. to run nova's pep8 tox env on wallaby i need to create an vm/container to do that because it wont actully work on a modern operating system. the system packages that we leverage are too new and cause issues.
We still have Ubuntu 20.04 among nodepool images for older releases, which can handle py3.8 nicely. So technically - there should be no issues in getting dependencies. And even if external dependencies will start dropping support - we have u-c specifically for this reason.
you have already been given that notice as we notified everyone that ubuntu 20.04 was last supported in 2023.1 an all othe supported distos in that release ship python 3.9+ debian 11 and centos 9 stream both already use python 3.9, ubuntu 22.04 ships 3.10
Was you? As I can technically run python 3.8 pretty much easily on one of the supported platforms. If I am opening PTI for 2023.2 (https://governance.openstack.org/tc/reference/runtimes/2023.2.html), what I'm told is: * I need to have ubuntu 22.04/Debian 11 * I need to have Python 3.10 or 3.9, but 3.8 is also a thing. It's not said anywhere that I can't do py3.8 for $reasons on Ubuntu 22.04, am I?
so this is not a issue for the removal of python 3.8 i dont know of any suppproted distribution where openstack 2023.2+ was shiped on py38 so keeping supprot in 2023.2 or 2024.1 seam academic to me. do we know of any operator or deployment that would actully deploy this way?
Have we _anywhere_ in our guidelines connected Python versions to distros? On the contrary, in "Extending support and testing for release with the newer disto version" I see: "When any release bumps the minimum supported distro platform OR python version", meaning these are 2 distinct things that are handled separately?
i think retoactivly adding platform and python version that were previously agreed to be drop undermines that platfrom and our ablity to maintain it. to be clear as a contibutoer and core review i felt underminded by the tc when python 3.8 was readded in may. and i felt like we sent the wrong message to operators because we said somethign was supproted when none of the testing runtim operationg systems supproted a unifed deployment of openstack on that versoin of python.
No, I don't think we were adding a platform retroactively? As of today, what I see and read makes me think that platform and python version are 2 distinct things. Despite that I know, that intention was slightly different, but that is what we need to deal with as of today. And I don't know how a regular operator should guess that we inside the community perceive this information differently. However, I fully understand your frustration that support was re-added. And I also agree that it was my fault as well to fail to see the absence of a process for removing older python versions when we were agreeing to remove it, which resulted in pretty much chaos once this became happening, as there was not any alignment on how to proceed. And I totally agree that the message sent was wrong. But it was already sent and I'm not sure it can be revoked as of today. So the best we can do now, IMO, is to focus on making up a clean and organized process for Python deprecation in 2024.2.
---- On Thu, 12 Oct 2023 05:02:11 -0700 Dmitriy Rabotyagov wrote ---
we drop supprot for py38 and hten re-added it because some project were not ready to drop it. i stongly belive that was a mistake.
Well, I'm not ready to judge if it was a mistake or not - there were good arguments from both sides. But from projects perspective (not libraries) it was pretty much in line to stop supporting it for 2023.2, yes.
if we dont drop supprot in 2024.1 then i think we must drop it in 2024.2 i dont think it would be reasonable to ask porject to continue to supprot it and i honestly dont think it is a reasonbale ask for 2024.1 or 2023.2 i was very unhappy with teh late readdtion of it in may after it was finally removed.
Yes, absolutely, it must go away in 2024.2 for sure.
1. Py3.8. is going to be supported by Ubuntu 20.04 until April 2025 at worst, and then there's also ESM till 2030. I bet RHEL has smth like that as well. So security backports for python packages and python itself is not really our pain. we do not use packages form the disto in our testing outside the standar libary and intepreter so it is a problem for the libs we use from pypi
But how much is the security of packages a concern inside a containerized CI, when we're talking about running unit testing only (and not dropping it from setup.cfg)?
2. We still can test things quite consistently, since we have all our dependencies maintained in upper-constraints.. Basically due to U-C I can build and install Ocata as of today with Py2.7 (I'm not saying it's good or bad, just stating the fact) not entirly. in ci or a contienr yes but some of our deps will not compile on newer operating systems. i.e. to run nova's pep8 tox env on wallaby i need to create an vm/container to do that because it wont actully work on a modern operating system. the system packages that we leverage are too new and cause issues.
We still have Ubuntu 20.04 among nodepool images for older releases, which can handle py3.8 nicely. So technically - there should be no issues in getting dependencies. And even if external dependencies will start dropping support - we have u-c specifically for this reason.
you have already been given that notice as we notified everyone that ubuntu 20.04 was last supported in 2023.1 an all othe supported distos in that release ship python 3.9+ debian 11 and centos 9 stream both already use python 3.9, ubuntu 22.04 ships 3.10
Was you? As I can technically run python 3.8 pretty much easily on one of the supported platforms. If I am opening PTI for 2023.2 (https://governance.openstack.org/tc/reference/runtimes/2023.2.html), what I'm told is: * I need to have ubuntu 22.04/Debian 11 * I need to have Python 3.10 or 3.9, but 3.8 is also a thing.
It's not said anywhere that I can't do py3.8 for $reasons on Ubuntu 22.04, am I?
Yes, also we need to keep in mind that testing runtime is minimum expectation of supported/tested distro or python versions. We might not be able to support more versions of distro but we can support/test more python versions which can be done on distro old version images or install manually in new version images. I also still feel dropping it in 2024.2 is better way where we give deprecation in 2024.1 (SLURP) and so that any upgrade from 2024.1->2024.2 or 2025.1 will not be issue. -gmann
so this is not a issue for the removal of python 3.8 i dont know of any suppproted distribution where openstack 2023.2+ was shiped on py38 so keeping supprot in 2023.2 or 2024.1 seam academic to me. do we know of any operator or deployment that would actully deploy this way?
Have we _anywhere_ in our guidelines connected Python versions to distros? On the contrary, in "Extending support and testing for release with the newer disto version" I see: "When any release bumps the minimum supported distro platform OR python version", meaning these are 2 distinct things that are handled separately?
i think retoactivly adding platform and python version that were previously agreed to be drop undermines that platfrom and our ablity to maintain it. to be clear as a contibutoer and core review i felt underminded by the tc when python 3.8 was readded in may. and i felt like we sent the wrong message to operators because we said somethign was supproted when none of the testing runtim operationg systems supproted a unifed deployment of openstack on that versoin of python.
No, I don't think we were adding a platform retroactively? As of today, what I see and read makes me think that platform and python version are 2 distinct things. Despite that I know, that intention was slightly different, but that is what we need to deal with as of today. And I don't know how a regular operator should guess that we inside the community perceive this information differently.
However, I fully understand your frustration that support was re-added. And I also agree that it was my fault as well to fail to see the absence of a process for removing older python versions when we were agreeing to remove it, which resulted in pretty much chaos once this became happening, as there was not any alignment on how to proceed.
And I totally agree that the message sent was wrong. But it was already sent and I'm not sure it can be revoked as of today. So the best we can do now, IMO, is to focus on making up a clean and organized process for Python deprecation in 2024.2.
---- On Wed, 11 Oct 2023 06:43:10 -0700 Brian Rosmaita wrote ---
I thought about this issue some more after yesterday's TC meeting, and tried to articulate why I think keeping python 3.8 support for 2024.1 is a bad idea on the patch:
https://review.opendev.org/c/openstack/governance/+/895160
I don't know if my argument there will change anyone's mind, but since a lot of people have already voted on the gerrit review, I wanted to make sure that people are at least aware of it, to wit:
I just don't see that py38 support for 2024.1 makes sense. It's not a default version in Ubuntu 22.04, Debian 12, Debian 11, CentOS Stream 9, or Rocky Linux 9, which are the distros specifically called out in the current 2024.1 PTI that this proposal is patching.
While we can use Ubuntu 20.04 to run unit tests, we can't run master (2024.1 development) devstack in it, so it doesn't seem to me that Ubuntu 20.04 is a distribution that we can feasibly use for meaningful 2024.1 testing. This implies, in my opinion, that there is a solid reason for dropping python 3.8 support in advance of it going EOL, as required by [0].
Looking at the Python Update Process resolution [1], python 3.8 does not meet the three criteria set out in the "Unit Tests" section:
1. it's not the latest version of Python 3 available in any distro we can feasibly use for testing 2. It's not the default in any of the Linux distros identified in the 2024.1 PTI 3. It isn't used to run integration tests at the beginning of the 2024.1 (Caracal) cycle
Add to that the fact that libraries are beginning to drop support [2], add further that py38 will go EOL roughly 6 months after the 2024.1 release (no more security updates), I don't see a reason to wait until a key library forces us to make a change during the development cycle. I'd prefer to do it now.
But this is what we agreed in the below change in policy to keep python min version as much as we can. - https://review.opendev.org/c/openstack/governance/+/882154 Again, this is very low expectation on keeping/testing python3.8 which is to run the unit or functional tests so that we make sure we do not break installation or code error for python3.8. This is not very costly for upstream but a good help for users on older python. I agree on the point that if we are not able to test it due to not available in our supported distro (I think focal will continue having it) or any external deps/lib hardly break us. But we do not have that situation yet and I will again suggest we can deal with that once it happen. -gmann
[0] https://governance.openstack.org/tc/reference/pti/python.html#specific-comma... [1] https://governance.openstack.org/tc/resolutions/20181024-python-update-proce... [2] https://review.opendev.org/c/openstack/requirements/+/884564
Again, this is very low expectation on keeping/testing python3.8 which is to run the unit or functional tests so that we make sure we do not break installation or code error for python3.8. This is not very costly for upstream but a good help for users on older python.
I'm going to be honest; I find more to be a reason to drop it; not as a reason to keep it. Taking on the downside of being stuck to python 3.8-supporting libraries without even gaining the upside of being able to know that you can run a fully tested OpenStack on Python 3.8 doesn't seem to be a good value to me. Thanks, Jay Faulkner P.S. Apologies to those who got this twice; I didn't include the list the first time.
On 10/11/23 2:30 PM, Ghanshyam Mann wrote: [snip]
But this is what we agreed in the below change in policy to keep python min version as much as we can. - https://review.opendev.org/c/openstack/governance/+/882154
I don't think dropping python 3.8 support for 2024.1 violates that agreement. Lines 35-38: Projects should avoid removing Python versions that have not reached `End Of Life <https://devguide.python.org/versions/>`_ without a solid reason. It is recommended to keep compatability with older Python versions as long as possible. What we're disagreeing about is what counts as "a solid reason". The only distro mentioned anywhere in the "Tested Runtimes for 2024.1" document [0] that has python 3.8 as the default is Ubuntu 20.04, and we can't run master (2024.1 development) devstack on it. That strikes me as "a solid reason" not to support python 3.8 in 2024.1. Adding the fact that python 3.8 will stop receiving security fixes within 6 months of the 2024.1 release makes this reason even more solid.
Again, this is very low expectation on keeping/testing python3.8 which is to run the unit or functional tests so that we make sure we do not break installation or code error for python3.8. This is not very costly for upstream but a good help for users on older python.
Sure, but the governance patch you reference above is not the only consideration. The "2018-10-24 Python Update Process" [1] TC resolution has not been superseded by any other resolution, and dropping python 3.8 support in 2024.1 does not violate the three criteria set out there. (With respect to criterion 2, Ubuntu 20.04 is *not* listed in the targeted distributions section of [0].) In any case, I don't think the proposal to drop python 3.8 support violates previous agreements by the TC. [0] https://governance.openstack.org/tc/reference/runtimes/2024.1.html [1] https://governance.openstack.org/tc/resolutions/20181024-python-update-proce...
---- On Fri, 13 Oct 2023 05:42:48 -0700 Brian Rosmaita wrote ---
On 10/11/23 2:30 PM, Ghanshyam Mann wrote: [snip]
But this is what we agreed in the below change in policy to keep python min version as much as we can. - https://review.opendev.org/c/openstack/governance/+/882154
I don't think dropping python 3.8 support for 2024.1 violates that agreement. Lines 35-38:
Projects should avoid removing Python versions that have not reached `End Of Life https://devguide.python.org/versions/>`_ without a solid reason. It is recommended to keep compatability with older Python versions as long as possible.
What we're disagreeing about is what counts as "a solid reason". The only distro mentioned anywhere in the "Tested Runtimes for 2024.1" document [0] that has python 3.8 as the default is Ubuntu 20.04, and we can't run master (2024.1 development) devstack on it. That strikes me as "a solid reason" not to support python 3.8 in 2024.1. Adding the fact that python 3.8 will stop receiving security fixes within 6 months of the 2024.1 release makes this reason even more solid.
I do not think so because previous to this project-team-guide change[1] we used to follow the policy what you described that "keep bumping the python min version which is available default on the supported distro LTS version" and we kept bumping python min version without any solid reason other than just because it is not available as default in any of the supported distro. Which end up as issue in last cycle and we changed our policy and updated the documentation of python PTI also. I am not sure if we keep referring the old resolution 2018 you mentioned or the updated PTI documentation. If you feel it has to be updated through the resolution then I am ok to propose change[1] (but this change was accepted by ~7-8 TC members which seems same as pass it as resolution) to resolution but honestly saying we do not need everything in resolution and policy documentation should be something we can refer. [1] https://review.opendev.org/c/openstack/governance/+/882154 -gmann
Again, this is very low expectation on keeping/testing python3.8 which is to run the unit or functional tests so that we make sure we do not break installation or code error for python3.8. This is not very costly for upstream but a good help for users on older python.
Sure, but the governance patch you reference above is not the only consideration. The "2018-10-24 Python Update Process" [1] TC resolution has not been superseded by any other resolution, and dropping python 3.8 support in 2024.1 does not violate the three criteria set out there. (With respect to criterion 2, Ubuntu 20.04 is *not* listed in the targeted distributions section of [0].)
In any case, I don't think the proposal to drop python 3.8 support violates previous agreements by the TC.
[0] https://governance.openstack.org/tc/reference/runtimes/2024.1.html [1] https://governance.openstack.org/tc/resolutions/20181024-python-update-proce...
On Fri, Oct 13, 2023, at 10:57 AM, Ghanshyam Mann wrote:
---- On Fri, 13 Oct 2023 05:42:48 -0700 Brian Rosmaita wrote ---
On 10/11/23 2:30 PM, Ghanshyam Mann wrote: [snip]
But this is what we agreed in the below change in policy to keep python min version as much as we can. - https://review.opendev.org/c/openstack/governance/+/882154
I don't think dropping python 3.8 support for 2024.1 violates that agreement. Lines 35-38:
Projects should avoid removing Python versions that have not reached `End Of Life https://devguide.python.org/versions/>`_ without a solid reason. It is recommended to keep compatability with older Python versions as long as possible.
What we're disagreeing about is what counts as "a solid reason". The only distro mentioned anywhere in the "Tested Runtimes for 2024.1" document [0] that has python 3.8 as the default is Ubuntu 20.04, and we can't run master (2024.1 development) devstack on it. That strikes me as "a solid reason" not to support python 3.8 in 2024.1. Adding the fact that python 3.8 will stop receiving security fixes within 6 months of the 2024.1 release makes this reason even more solid.
I do not think so because previous to this project-team-guide change[1] we used to follow the policy what you described that "keep bumping the python min version which is available default on the supported distro LTS version" and we kept bumping python min version without any solid reason other than just because it is not available as default in any of the supported distro. Which end up as issue in last cycle and we changed our policy and updated the documentation of python PTI also.
There are two different but related issues here. First is having overlap with python versions across releases. I think it was TripleO that ran into trouble with a lack of overlap. We have this. python3.8 has been supported for years so users have many versions on which they can convert with overlap. This is something we should keep in mind for future versions though. The second is the one that you are referring to here and the underlying issue was out of order operations in removing support. Support was removed from libraries first. Libraries need to remove support last. Doing it out of order broke downstream consumers of the libraries. In both situations I'm sure we can coordinate and plan ahead better. I wouldn't characterize these as upgrades simply because no current LTS supports them. As I wrote in a previous email python3.8 will be EOL early in the lifetime of 2024.1. Historically we've seen this lead to problems. I think it is better for users to get a clear signal from us that we would prefer to avoid those issues entirely and have them update sooner. This is a solid reason in my opinion. If we need additional reasons Python 3.10 and 3.11 have improved performance. In theory, the sooner we drop older slower pythons the sooner we get faster gate round trips, more performant API servers, and so on. I have no profiled any of the OpenStack services to say what the improvement will be (if any), but we have seen significant improvements in Zuul.
I am not sure if we keep referring the old resolution 2018 you mentioned or the updated PTI documentation. If you feel it has to be updated through the resolution then I am ok to propose change[1] (but this change was accepted by ~7-8 TC members which seems same as pass it as resolution) to resolution but honestly saying we do not need everything in resolution and policy documentation should be something we can refer.
[1] https://review.opendev.org/c/openstack/governance/+/882154
-gmann
participants (8)
-
Brian Rosmaita
-
Clark Boylan
-
Dmitriy Rabotyagov
-
Ghanshyam Mann
-
Jay Faulkner
-
Kurt Garloff
-
smooney＠redhat.com
-
Sławek Kapłoński