[all] Debian unstable has Python 3.11: please help support it.
Hi everyone! I know the usual answer: "we don't do that", or "this is unsupported in current version", however, as always, that's the way things are: OpenStack isn't alone living in Debian, and Debian Unstable now has Python 3.11, which isn't going to change simply because the OpenStack community decides that "it's not supported". So it'd be nice if we could support it. I'm quite sure that, as usual, the upload of a new Python interpreter version will break my world. I'll try to summit patches when I can, but I also expect help if possible. The challenge is: Debian Bookworm will be shipping Zed and Python 3.11, most likely... Cheers, Thomas Goirand (zigo)
Hi everyone!
I know the usual answer: "we don't do that", or "this is unsupported in current version", however, as always, that's the way things are: OpenStack isn't alone living in Debian, and Debian Unstable now has Python 3.11, which isn't going to change simply because the OpenStack community decides that "it's not supported".
So it'd be nice if we could support it. I'm quite sure that, as usual, the upload of a new Python interpreter version will break my world. I'll try to summit patches when I can, but I also expect help if possible.
The challenge is: Debian Bookworm will be shipping Zed and Python 3.11, most likely...
On Thu, 2022-07-14 at 15:38 +0200, Thomas Goirand wrote: thanks for the heads up. last time i tried using bookworm i had to force 3.9 via a virual enve to fully deploy a working openstack. i know that you and the eventlet folk have actuly resolved the issue i hit already so 3.10 shoudl work now but i suspect evently will be the most fragil thing with getting 3.11 to work. in terms of offical testing runtime you are correct that "this is unsupported in current version" since we determin the supported/tested interperters for a release at the start fo the cycle. and currenlty we only test 3.8 and 3.9 with experimatal support for 3.10 im not against reviewing/merging patches that are needed for 3.11 as long as we do not break 3.8 for next cycle im hoping we will drop ubuntu 20.04 in favor of ubuntu 22.04 and we can rais our min version to 3.9 and add 3.10 and 3.11 support formally to our testing runtimes if those are aviable to test with in lts releases such as debian 11 or ubuntu 22.04 so making a start on that in zed on a best effort baisis i think makes sense. the only thing is that if we have to choose betten 3.8 support and 3.11 we need to ensure we maintian the agreed runtime supprot but i doubt we will need to make that choice. do we currently have 3.11 aviable in any of the ci images? i belive we have 22.04 image aviable is it installbale there or do we have debian bookworm images we can use to add a non voting tox py311 job to the relevent project repos?
Cheers,
Thomas Goirand (zigo)
On 2022-07-14 15:01:14 +0100 (+0100), Sean Mooney wrote: [...]
do we currently have 3.11 aviable in any of the ci images? i belive we have 22.04 image aviable is it installbale there or do we have debian bookworm images we can use to add a non voting tox py311 job to the relevent project repos?
Not to my knowledge, no. Ubuntu inherits most of their packages from Debian, which has only just added a Python 3.11 pre-release, so it will take time to end up even in Ubuntu under development (Ubuntu Kinetic which is slated to become 22.10 still only has python3.10 packages for the moment). It's probable they'll backport a python3.11 package to Jammy once available, though there's no guarantee, and based on historical backports it probably won't be until upstream 3.11.1 is tagged at the very earliest. Keep in mind that what Debian has at the moment is a package of Python 3.11.0b4, since 3.11.0 isn't even scheduled for an upstream release until October (two days before we're planning to release OpenStack Zed). Further, it's not even in Debian bookworm yet, and it's hard to predict how soon it will be able to transition out of unstable either. Let's be clear, what's being asked here is that OpenStack not just test against the newest available Python release, but in fact to continually test against pre-releases of the next Python while it's still being developed. While I understand that this would be nice, I hardly think it's a reasonable thing to expect. We have a hard enough time just keeping up with actual releases of Python which are current at the time we start a development cycle. -- Jeremy Stanley
On Thu, Jul 14, 2022, at 7:30 AM, Jeremy Stanley wrote:
On 2022-07-14 15:01:14 +0100 (+0100), Sean Mooney wrote: [...]
do we currently have 3.11 aviable in any of the ci images? i belive we have 22.04 image aviable is it installbale there or do we have debian bookworm images we can use to add a non voting tox py311 job to the relevent project repos?
Not to my knowledge, no. Ubuntu inherits most of their packages from Debian, which has only just added a Python 3.11 pre-release, so it will take time to end up even in Ubuntu under development (Ubuntu Kinetic which is slated to become 22.10 still only has python3.10 packages for the moment). It's probable they'll backport a python3.11 package to Jammy once available, though there's no guarantee, and based on historical backports it probably won't be until upstream 3.11.1 is tagged at the very earliest.
Keep in mind that what Debian has at the moment is a package of Python 3.11.0b4, since 3.11.0 isn't even scheduled for an upstream release until October (two days before we're planning to release OpenStack Zed). Further, it's not even in Debian bookworm yet, and it's hard to predict how soon it will be able to transition out of unstable either.
Let's be clear, what's being asked here is that OpenStack not just test against the newest available Python release, but in fact to continually test against pre-releases of the next Python while it's still being developed. While I understand that this would be nice, I hardly think it's a reasonable thing to expect. We have a hard enough time just keeping up with actual releases of Python which are current at the time we start a development cycle.
Big ++ on this last point. I think right now the most important thing you can do to ensure the transition to python3.11 goes as smoothly as possible is to shore up and improve the current python3.10 testing.
-- Jeremy Stanley
Attachments: * signature.asc
On 7/14/22 16:30, Jeremy Stanley wrote:
Let's be clear, what's being asked here is that OpenStack not just test against the newest available Python release, but in fact to continually test against pre-releases of the next Python while it's still being developed.
I'm not asking for that. :) All what I'm asking, is that when Python RC releases are out, and I report a bug, the community has the intention to fix it as early as possible, at least in master (and maybe help with backports if it's very tricky: I can manage trivial backporting by myself). That's enough for me, really (at least it has been enough in the past...).
We have a hard enough time just keeping up with actual releases of Python which are current at the time we start a development cycle.
Yeah, though it'd be nice if we could have the latest interpreter in use in Unstable for a non-voting job, starting when the interpreter is released (or at least when the first RCs are out). We discussed this already, didn't we? I know it's a "would be nice thing" and that nobody has time to work on this... :/ Cheers, Thomas Goirand (zigo)
On 2022-07-15 12:05:31 +0200 (+0200), Thomas Goirand wrote: [...]
All what I'm asking, is that when Python RC releases are out, and I report a bug, the community has the intention to fix it as early as possible, at least in master (and maybe help with backports if it's very tricky: I can manage trivial backporting by myself). That's enough for me, really (at least it has been enough in the past...). [...]
I don't think fixes have been refused in the past simply because they address a problem observed with a newer Python interpreter than we test on. Just be aware that when it comes to pre-merge testing of proposed changes for OpenStack, the time to decide what platforms and interpreter versions we'll test with is at the end of the previous cycle. For Zed that's Python 3.8 and 3.9, though projects are encouraged to also try 3.10 if they can (we got the ability to test that partway into the development cycle). We have to set these expectations before we begin work on a new version of OpenStack, so that we don't change our testing goals for developers while they're in the middle of trying to write the software. -- Jeremy Stanley
On Fri, Jul 15, 2022 at 1:11 PM Jeremy Stanley <fungi@yuggoth.org> wrote:
On 2022-07-15 12:05:31 +0200 (+0200), Thomas Goirand wrote: [...]
All what I'm asking, is that when Python RC releases are out, and I report a bug, the community has the intention to fix it as early as possible, at least in master (and maybe help with backports if it's very tricky: I can manage trivial backporting by myself). That's enough for me, really (at least it has been enough in the past...). [...]
I don't think fixes have been refused in the past simply because they address a problem observed with a newer Python interpreter than we test on. Just be aware that when it comes to pre-merge testing of proposed changes for OpenStack, the time to decide what platforms and interpreter versions we'll test with is at the end of the previous cycle. For Zed that's Python 3.8 and 3.9, though projects are encouraged to also try 3.10 if they can (we got the ability to test that partway into the development cycle). We have to set these expectations before we begin work on a new version of OpenStack, so that we don't change our testing goals for developers while they're in the middle of trying to write the software. -- Jeremy Stanley
Just to add to the "what python version where" discussion, I wanted to add the situation with Ubuntu packages and distro versions: As a quick summary for the Ubuntu cloud archive and distro packages for OpenStack, for yoga we ship the packages for 20.04 LTS (focal) - in the focal-yoga cloud archive [1] and 22.04 LTS (jammy) as part of distro. 20.04 LTS (focal) shipped with py3.8 so the packages work with that, but 22.04 LTS (jammy) shipped with py3.10, which means that (ultimately) we try very hard to get all the Ubuntu OpenStack yoga packages to work on py3.10. Ubuntu Kinetic (22.10) currently looks like it will also ship with py3.10 [2]. OpenStack Zed is 'partnered' with 22.10/kinetic as the Zed packages will be shipped in distro for 22.10/kinetic and as jammy-kinetic in the cloud archive. Therefore, we'll also be supporting py3.10 for Zed. py3.11 is slated for release on 3rd October 2022, so it won't make Kinetic (with luck), but it will almost certainly make "L" (23.04), which will be against (for us) OpenStack "A". I guess this is going to be quite challenging for everyone! Why is it like this? Well, the latest current stable version of python is selected for each distro version to ensure that each LTS is running the latest stable version; thus each interim updates so that it is easier to work towards the next LTS stable release. Thus, for the Ubuntu packages, we have to support that version for each of the OpenStack releases during the LTS (e.g. for jammy, py3.10) for Y->B versions of OpenStack. So as a summary (for the Ubuntu packages/python versions): - Yoga: py3.8: focal-yoga=py3.8, py3.10: jammy (distro) - Zed: py3.10 (jammy-zed, kinetic(distro)) - "A": py3.10 (jammy-"A"), py3.11 ("L"/22.10 (distro)) - "B": py3.10 (jammy-"B"), py3.11 (py3.12?) - etc Essentially, for each LTS release (every 2 years, April), we eventually ship 4 versions of OpenStack, one with the distro packages, and 3 in the cloud archive. Also, each interim release of Ubuntu also gets a corresponding OpenStack release set of packages for the python version in that release. We do additional testing (which we call distro-regression) where we check the packages against each version of Ubuntu and run tempest to "ensure" that they work. Hope this is useful. [1] http://ubuntu-cloud.archive.canonical.com/ubuntu/dists/focal-updates/yoga/ [2] https://packages.ubuntu.com/kinetic/python/ -- Alex Kavanagh - Software Engineer OpenStack Engineering - Data Centre Development - Canonical Ltd
On Fri, Jul 15, 2022 at 1:11 PM Jeremy Stanley <fungi@yuggoth.org> wrote:
On 2022-07-15 12:05:31 +0200 (+0200), Thomas Goirand wrote: [...]
All what I'm asking, is that when Python RC releases are out, and I report a bug, the community has the intention to fix it as early as possible, at least in master (and maybe help with backports if it's very tricky: I can manage trivial backporting by myself). That's enough for me, really (at least it has been enough in the past...). [...]
I don't think fixes have been refused in the past simply because they address a problem observed with a newer Python interpreter than we test on. Just be aware that when it comes to pre-merge testing of proposed changes for OpenStack, the time to decide what platforms and interpreter versions we'll test with is at the end of the previous cycle. For Zed that's Python 3.8 and 3.9, though projects are encouraged to also try 3.10 if they can (we got the ability to test that partway into the development cycle). We have to set these expectations before we begin work on a new version of OpenStack, so that we don't change our testing goals for developers while they're in the middle of trying to write the software. -- Jeremy Stanley
Just to add to the "what python version where" discussion, I wanted to add the situation with Ubuntu packages and distro versions:
As a quick summary for the Ubuntu cloud archive and distro packages for OpenStack, for yoga we ship the packages for 20.04 LTS (focal) - in the focal-yoga cloud archive [1] and 22.04 LTS (jammy) as part of distro. 20.04 LTS (focal) shipped with py3.8 so the packages work with that, but 22.04 LTS (jammy) shipped with py3.10, which means that (ultimately) we try very hard to get all the Ubuntu OpenStack yoga packages to work on py3.10.
Ubuntu Kinetic (22.10) currently looks like it will also ship with py3.10 [2]. OpenStack Zed is 'partnered' with 22.10/kinetic as the Zed packages will be shipped in distro for 22.10/kinetic and as jammy-kinetic in the cloud archive. Therefore, we'll also be supporting py3.10 for Zed. py3.11 is slated for release on 3rd October 2022, so it won't make Kinetic (with luck), but it will almost certainly make "L" (23.04), which will be against (for us) OpenStack "A". I guess this is going to be quite challenging for everyone!
Why is it like this? Well, the latest current stable version of python is selected for each distro version to ensure that each LTS is running the latest stable version; thus each interim updates so that it is easier to work towards the next LTS stable release. Thus, for the Ubuntu packages, we have to support that version for each of the OpenStack releases during the LTS (e.g. for jammy, py3.10) for Y->B versions of OpenStack.
So as a summary (for the Ubuntu packages/python versions):
- Yoga: py3.8: focal-yoga=py3.8, py3.10: jammy (distro) - Zed: py3.10 (jammy-zed, kinetic(distro)) - "A": py3.10 (jammy-"A"), py3.11 ("L"/22.10 (distro)) - "B": py3.10 (jammy-"B"), py3.11 (py3.12?) - etc
Essentially, for each LTS release (every 2 years, April), we eventually ship 4 versions of OpenStack, one with the distro packages, and 3 in the cloud archive. Also, each interim release of Ubuntu also gets a corresponding OpenStack release set of packages for the python version in that release. We do additional testing (which we call distro-regression) where we check the packages against each version of Ubuntu and run tempest to "ensure" that they work.
Hope this is useful.
On Tue, 2022-07-19 at 20:57 +0100, Alex Kavanagh wrote: that is not as missalinged with our testing as debain will be. currently our ubuntu testing is based on 20.04 and we have 22.04 image which we use for the non voting python 3.10 jobs (orginaly those were fedora but we change when ubuntu 22.04 became avaiable) for the A release the testing runtimes have not been chossen yet but my expecation is we will drop 20.04 form the testing requirements for master and move to 22.04 making 3.10 our default python for tempest integration testing. assuming the python 3.11 interperter and ideally standard libary are aviabel as a package to install on 22.04 we should be able to add a non vovting py3.11 tox job to replace our current py3.10 job and perhaps even have a periodic-weekly python 3.11 tempest job. while the master branch for A will open in advance of the offical release the end of the zed schdule is the week of october 3rd. https://releases.openstack.org/zed/schedule.html the A branches technially will open on september 15th with the release of rc1 so if the py3.11 package is aviable close to the offial start of the A cycle we may be able to include it in the project testing requriementes even if tis only an advisery testing target and not a required one. for B we shoudl have it as a required testing target but we will need to keep py3.9 likely until C or D since centos/rhel 9 will still be shiping that as the only supproted interperter. im not sure if ubuntu has the same poicly where non default intpererteres are shipped only for dev and testing use and not for production. centos/rhel also only ships one version fo the standard libary so new libiary feature would not be avaiabel with the new inteperter if i understand correctly. as a result of needing to support py 3.9 that means we cannot use any libeary feature that has not either been backported to 3.9 though standalone packages as dataclaess was for py3.6. the main concen this bring is we also need to be carful not to continue using any py 3.9 feature that are removed form 3.11 or 3.12 ectra.
[1] http://ubuntu-cloud.archive.canonical.com/ubuntu/dists/focal-updates/yoga/ [2] https://packages.ubuntu.com/kinetic/python/
On 2022-07-20 09:34:18 +0100 (+0100), Sean Mooney wrote: [...]
for the A release the testing runtimes have not been chossen yet but my expecation is we will drop 20.04 form the testing requirements for master and move to 22.04 making 3.10 our default python for tempest integration testing.
Unless I've misunderstood the recent TC discussions around supported platform overlap, the new release cadence, and upgrade testing, it's my understanding that we'll still at least need to test that we can upgrade from Zed to Anchovy/Anteater/Antelope on Ubuntu 20.04 LTS (future A->B and A->C upgrade testing will be able to just use 22.04 LTS though), so it's not going away entirely for master branch tests until B.
assuming the python 3.11 interperter and ideally standard libary are aviabel as a package to install on 22.04 we should be able to add a non vovting py3.11 tox job to replace our current py3.10 job and perhaps even have a periodic-weekly python 3.11 tempest job. [...]
In the past, Ubuntu has waited to backport a new Python minor version until after its .1 point release is available. For 3.11.1 that's scheduled to be approximately 2 months after 3.11.0 is done, so expect early December. That puts any Jammy backport of it into 2023 at the earliest, I expect, at best a couple of months before we release so probably not soon enough in the cycle for meaningful testing before master is open for B cycle work. -- Jeremy Stanley
On Wed, 2022-07-20 at 11:49 +0000, Jeremy Stanley wrote:
On 2022-07-20 09:34:18 +0100 (+0100), Sean Mooney wrote: [...]
for the A release the testing runtimes have not been chossen yet but my expecation is we will drop 20.04 form the testing requirements for master and move to 22.04 making 3.10 our default python for tempest integration testing.
Unless I've misunderstood the recent TC discussions around supported platform overlap, the new release cadence, and upgrade testing, it's my understanding that we'll still at least need to test that we can upgrade from Zed to Anchovy/Anteater/Antelope on Ubuntu 20.04 LTS ah correct this is not really related to the new lifecycle. we will need to test with ubuntu 20.04 for grenade or other upgrade jobs since we do not update the os in those jobs so deploy on the older release. (future A->B and A->C upgrade testing will be able to just use 22.04 LTS though), so it's not going away entirely for master branch tests until B. yes 22.04 can only become the base of the grenade job for A->B all the standard jobs should mvoe to 22.04 in B ideally which means we need to keep 3.8 until B since that is the default in 20.04 in B we can increase the minium to 3.9 and add 3.11 based on the timeline you explain below since that would better align with our developemnt cycle.
assuming the python 3.11 interperter and ideally standard libary are aviabel as a package to install on 22.04 we should be able to add a non vovting py3.11 tox job to replace our current py3.10 job and perhaps even have a periodic-weekly python 3.11 tempest job. [...]
In the past, Ubuntu has waited to backport a new Python minor version until after its .1 point release is available. For 3.11.1 that's scheduled to be approximately 2 months after 3.11.0 is done, so expect early December. That puts any Jammy backport of it into 2023 at the earliest, I expect, at best a couple of months before we release so probably not soon enough in the cycle for meaningful testing before master is open for B cycle work. ack so ya that likely means it cant be considerd a required target for A but could be optioanlly tested by some projects once its avaiable. is assume there is no reason not to add 3.10 to the requried testing runtimes for A based on ubuntu 22.04 form its current best effort status.
On 7/14/22 16:01, Sean Mooney wrote:
do we currently have 3.11 aviable in any of the ci images? i belive we have 22.04 image aviable is it installbale there or do we have debian bookworm images we can use to add a non voting tox py311 job to the relevent project repos?
Hi, Currently, we only have Python 3.11 beta 4 (ie: 3.11.0~b4-1) available in Debian Unstable. It wont be available in Bookworm until the python 3.10 -> 3.11 transition is over in Debian Unstable. During this process, Python 3.11 will only be an available Python version, but not the default. It will then become the default Python 3, and then, Python 3.10 will be removed from Unstable. THEN Python 3.11 will be fully the Bookworm version. This probably will take a few months. FYI, I very much know the patches will be done on a best effort basis only. I'm fine with that, and I'm used to discuss it with the community, and do backports of patches that land in master. My mail was just a call to the community so that we keep in mind that it's coming. I have no idea what the breakages will be (yet), but I'm looking forward figuring it out. Over the years, I kind of have fun doing so, even if I still think breaking the world every few months is a terrible idea. In a more general way, I am convince that it's always best for all of us if we can find a way to test with the latest everything, including the interpreter. Waiting for Ubuntu to have the latest interpreter is IMO broken by design, because the Python version transition always happen in Debian Unstable first (and made by the same person that maintains the Python interpreter in both Debian and Ubuntu: Matthias Klose, aka doko). Not only for the interpreter, if we could find a way to test things in Debian Unstable, always, as non-voting jobs, we would see the failures early. I'd love we he had such a non-voting job, that would also use the latest packages from PyPi, just so that we could at least know what will happen in a near future. Your thoughts everyone? Cheers, Thomas Goirand (zigo)
On 2022-07-15 11:56:25 +0200 (+0200), Thomas Goirand wrote: [...]
Not only for the interpreter, if we could find a way to test things in Debian Unstable, always, as non-voting jobs, we would see the failures early. I'd love we he had such a non-voting job, that would also use the latest packages from PyPi, just so that we could at least know what will happen in a near future.
Your thoughts everyone?
My thought is that there is some irony in the timing of your question, since just yesterday[*] the OpenStack Technical Committee seems to have reached a consensus that CentOS is no longer a stable enough platform for pre-merge testing. Not necessarily anything to do with running OpenStack on it specifically, but just that generally it's the place where minimally-tested things go to see if there are any problems with them before they wind up in RHEL. That's a pretty close parallel to Debian's unstable/testing distributions as the place where problems get worked out before they get into the next stable release. Taking things from an OpenDev Collaboratory perspective, we've struggled to keep images for frequently-changing distros like Fedora, Gentoo, or OpenSUSE Tumbleweed working at all because things frequently change which break our ability to build or boot those images. More static distributions like Debian stable, Ubuntu LTS, or CentOS (before it became Stream), have a much less frequent update cycle and so are far easier for us to plan for and stay on top of. [*] https://meetings.opendev.org/meetings/tc/2022/tc.2022-07-14-15.00.log.html#l... -- Jeremy Stanley
On Fri, 15 Jul 2022 at 11:59, Thomas Goirand <zigo@debian.org> wrote:
Not only for the interpreter, if we could find a way to test things in Debian Unstable, always, as non-voting jobs, we would see the failures early. I'd love we he had such a non-voting job, that would also use the latest packages from PyPi, just so that we could at least know what will happen in a near future.
Well, we can have periodic and experimental, master-only jobs to test things on Debian unstable because it's always interesting to see the upcoming breakage (or better yet - be able to pinpoint it to a certain change happening in Debian unstable that caused it). The job would only utilise the interpreter and the helper binaries (like ovs) - all targets I can think of are capable only of using pip-installed deps and not Debian packages so that part we cannot really cover reliably at all. If that makes sense to you, we can work towards that direction. The biggest issue will still be the bootability/usability of the infra image though. -yoctozepto
On Fri, 2022-07-15 at 14:44 +0200, Radosław Piliszek wrote:
On Fri, 15 Jul 2022 at 11:59, Thomas Goirand <zigo@debian.org> wrote:
Not only for the interpreter, if we could find a way to test things in Debian Unstable, always, as non-voting jobs, we would see the failures early. I'd love we he had such a non-voting job, that would also use the latest packages from PyPi, just so that we could at least know what will happen in a near future.
Well, we can have periodic and experimental, master-only jobs to test things on Debian unstable because it's always interesting to see the upcoming breakage (or better yet - be able to pinpoint it to a certain change happening in Debian unstable that caused it).
for the nova ci we have moved centos 9 stream jobs to the periodic-weekly pipeline and we are going to monitor it in our weekly meeting. i dont really have any objection to adding a debian testing or debian stable based job there too. be it in the form of a tempest job or tox job. we dont really have the capsity either in review bandwith or ci resouce to have enstable distros in a voting or non voting capsity on every patch i.e. check and gate pipelines. but a weekly periodic its proably doable. one thing we have to be carful of however is a concern i raised last cycle with extending 3.6 support. some of the 3.6 deprecated fature were drop in 3.10 and more feature that were depfreced in later releases are droped in 3.11. while we can try and support the newr releases 3.9 compatiable will be needed for a long time. so if 3.11 or 3.12 is fundementaly in compatibel with 3.9 due to a libary change ectra that will be problematic sicne cento 9 derived distro will be on 3.9 for several years to come. i dont actully know the exact lifecycle uels for how often new centos/rhel majory reseas happen but its usuarly aroud the .6 release of the current release that the .0 release of the next majory version happen so every 5 years or so. i really dont know the plans for rhel 10 but for rhel9/centos 9 lifetime 3.9 will be the python version used so we need to balnce fast moving distors like debian and slow moving distors like centos and enable both. that might mean we will ahve to drop some deps, avoid some features or utilise compatablity libs in some cases similar to six or mock the lib vs unittest.mock so when fixing any 3.11 incompatiablity we need to still maintain 3.8 compatiablity for zed and 3.9 compatiblity in AA+ i woudl guess it will be the CC or DD release before we coudl consdier droping 3.9 suppot but i think we could drop 3.8 in AA.
The job would only utilise the interpreter and the helper binaries (like ovs) - all targets I can think of are capable only of using pip-installed deps and not Debian packages so that part we cannot really cover reliably at all. If that makes sense to you, we can work towards that direction. The biggest issue will still be the bootability/usability of the infra image though.
-yoctozepto
On Fri, Jul 15, 2022, at 7:14 AM, Sean Mooney wrote:
On Fri, 2022-07-15 at 14:44 +0200, Radosław Piliszek wrote:
On Fri, 15 Jul 2022 at 11:59, Thomas Goirand <zigo@debian.org> wrote:
Not only for the interpreter, if we could find a way to test things in Debian Unstable, always, as non-voting jobs, we would see the failures early. I'd love we he had such a non-voting job, that would also use the latest packages from PyPi, just so that we could at least know what will happen in a near future.
Well, we can have periodic and experimental, master-only jobs to test things on Debian unstable because it's always interesting to see the upcoming breakage (or better yet - be able to pinpoint it to a certain change happening in Debian unstable that caused it).
for the nova ci we have moved centos 9 stream jobs to the periodic-weekly pipeline and we are going to monitor it in our weekly meeting. i dont really have any objection to adding a debian testing or debian stable based job there too. be it in the form of a tempest job or tox job.
we dont really have the capsity either in review bandwith or ci resouce to have enstable distros in a voting or non voting capsity on every patch i.e. check and gate pipelines. but a weekly periodic its proably doable.
one thing we have to be carful of however is a concern i raised last cycle with extending 3.6 support.
some of the 3.6 deprecated fature were drop in 3.10 and more feature that were depfreced in later releases are droped in 3.11. while we can try and support the newr releases 3.9 compatiable will be needed for a long time. so if 3.11 or 3.12 is fundementaly in compatibel with 3.9 due to a libary change ectra that will be problematic sicne cento 9 derived distro will be on 3.9 for several years to come.
Removals for 3.10 are captured here: https://docs.python.org/3/whatsnew/3.10.html#removed and for 3.11 at https://docs.python.org/3.11/whatsnew/3.11.html#removed if anyone is curious. Considering the number of projects that appear to be currently running python3.8 and 3.10 job successfully, the major problem here is likely going to be if/when our dependencies decide to stop supporting older pythons. Often times we can get away with simply having different requirements for different versions of python, but that may not always work for each dependency.
i dont actully know the exact lifecycle uels for how often new centos/rhel majory reseas happen but its usuarly aroud the .6 release of the current release that the .0 release of the next majory version happen so every 5 years or so. i really dont know the plans for rhel 10 but for rhel9/centos 9 lifetime 3.9 will be the python version used so we need to balnce fast moving distors like debian and slow moving distors like centos and enable both. that might mean we will ahve to drop some deps, avoid some features or utilise compatablity libs in some cases similar to six or mock the lib vs unittest.mock
so when fixing any 3.11 incompatiablity we need to still maintain 3.8 compatiablity for zed and 3.9 compatiblity in AA+
i woudl guess it will be the CC or DD release before we coudl consdier droping 3.9 suppot but i think we could drop 3.8 in AA.
Python 3.9 EOL is October 2025. With 3.6 we saw many libraries maintain compatibility until the EOL for that version. I'm hopeful that trend will continue and this will largely be a non issue for us.
The job would only utilise the interpreter and the helper binaries (like ovs) - all targets I can think of are capable only of using pip-installed deps and not Debian packages so that part we cannot really cover reliably at all. If that makes sense to you, we can work towards that direction. The biggest issue will still be the bootability/usability of the infra image though.
-yoctozepto
participants (6)
-
Alex Kavanagh
-
Clark Boylan
-
Jeremy Stanley
-
Radosław Piliszek
-
Sean Mooney
-
Thomas Goirand