Re: [all][tc] Minium python version for yoga and centos 9 stream.
resending to correct list On Tue, 2021-09-07 at 10:51 +0100, Sean Mooney wrote:
hi i was jsut looking at the yoga runtimes an notice that we have not really updated it with regards to minium python versions.
currently
the runtime defintion is as follows https://github.com/openstack/governance/blob/master/reference/runtimes/yoga....
It is the :doc:`policy <../../resolutions/20181024-python-update-process>` that each OpenStack release cycle will target the latest available version of Python; default Python runtimes on the distributions listed above; and versions used in integration tests at the start of the cycle, at least until the point when all projects have migrated to a later version.
Based on the criteria above, all Python-based projects must target and test against, at a minimum:
Python 3.6 (default in CentOS 8 Stream) Python 3.8 (latest available; default in Ubuntu 20.04)
looking at to look at https://repology.org/project/python/versions what python version are shpping by default now in many distro more and more of them are moving to python 3.9 which is currently not in the test list and wer are starting to fall quite far behind
debian 11/12/stable are currently using 3.9.2 centos 8 defaults to 3.6.8 but ships 3.8.6 and 3.9.2 as appstreams centos 9 stream will default to python3-3.9.6-6.el9.x86_64.rpm https://composes.stream.centos.org/test/latest-CentOS-Stream/compose/BaseOS/... ubuntu 20.04 default to 3.8.2 but also ships 3.9.5 in universe ubuntu 21.04 default to 3.9.5 so we can expect that when yoga ships on ubuntu 22.04 it will be based on at least python 3.9 and proably 3.10 fedora well they basically ship everythin but i belive 3.9 is not the default based on complaints some of my fellow contiutors had with runnign unit and functional tests.
openSUSE Leap 15.3 main/oss python39 3.9.4 ship 3.9 but default to 3.6 they aslo nolonger have an openstack product but it would still be nice to keep them in mind.
over all i think we shoudl consider raising or min tested/supported python version to python 3.8 for yoga and consider updating from centos 8 to centos 9. i also think it worth including python 3.9 in the required testing versions for python projects.
several distros will be releaseing openstack xena/wallaby with python 3.9 and also yoga likely with 3.10 in some cases. if we only offically test with 3.6 and 3.8 we could have some hard probalems to solve going forward. the optional testing with unit and functional tests that some project have for python 3.9 certenly helps but actully having an integrated tempest gate job that runs on 3.9 i think is going to be increasingly important.
readring centos 9 stream, it was orginally planned of release at the end of q2. now that date has obviosly passed but i know that they are still hopeful to have a release ready in the next month or so. cloud images are avaibel at https://composes.stream.centos.org/production/latest-CentOS-Stream/compose/B... although the repos are still only partly complete https://composes.stream.centos.org/production/latest-CentOS-Stream/compose/B... and there are currently no active mirros so its still officaly not released https://www.centos.org/centos-stream/ and i don not have any ETA beyond soonTM. as such im aware we cant offically declare centos 9 stream as part of the tested runtimes at this point but it might be at least worht listing it as an experimtal runtime as i suspec there will be a desire to swtich to it during the yoga cycle for some proejct. ooo/rdo come to mind as they discussed this for xena but obviously that was not possibel since it was not released in time.
regards sean.
ps removing python 3.6 and replacing it with 3.9 instead of just adding 3.9 as an addtional runtime is more to manage our testing there isnt nessarly any python feature that i explcitly would like to be able to use that are added in 3.8 that would be enabled by making that the new min python verion that is motivating this suggestion. there are some nice thingin 3.9/3.10 such as pattern mataching that are commign but it will be some time (2-4 years)before we can actully use those as our min version so i was just being practial and suggestion we raise to 3.8 and add 3.9 as required with 3.10 as optional/experimal for those that want it.
On Tue, Sep 7, 2021 at 1:10 PM Sean Mooney <smooney@redhat.com> wrote:
hi i was jsut looking at the yoga runtimes an notice that we have not really updated it with regards to minium python versions.
currently
the runtime defintion is as follows https://github.com/openstack/governance/blob/master/reference/runtimes/yoga....
It is the :doc:`policy <../../resolutions/20181024-python-update-process>` that each OpenStack release cycle will target the latest available version of Python; default Python runtimes on the distributions listed above; and versions used in integration tests at
resending to correct list On Tue, 2021-09-07 at 10:51 +0100, Sean Mooney wrote: the start of the cycle, at least until the point when all projects have migrated to a later version.
Based on the criteria above, all Python-based projects must target
and test against, at a minimum:
Python 3.6 (default in CentOS 8 Stream) Python 3.8 (latest available; default in Ubuntu 20.04)
looking at to look at https://repology.org/project/python/versions what
more and more of them are moving to python 3.9 which is currently not in
python version are shpping by default now in many distro the test list and wer are starting to fall quite far behind
debian 11/12/stable are currently using 3.9.2 centos 8 defaults to 3.6.8 but ships 3.8.6 and 3.9.2 as appstreams
We shouldn't rely on non-default versions.
centos 9 stream will default to python3-3.9.6-6.el9.x86_64.rpm https://composes.stream.centos.org/test/latest-CentOS-Stream/compose/BaseOS/... ubuntu 20.04 default to 3.8.2 but also ships 3.9.5 in universe
Same.
ubuntu 21.04 default to 3.9.5 so we can expect that when yoga ships on ubuntu 22.04 it will be based on at least python 3.9 and proably 3.10 fedora well they basically ship everythin but i belive 3.9 is not the default based on complaints some of my fellow contiutors had with runnign unit and functional tests.
openSUSE Leap 15.3 main/oss python39 3.9.4 ship 3.9 but default to 3.6 they aslo nolonger have an openstack product but it would still be nice to keep them in mind.
Yep, some of us (namely Bifrost) support openSUSE Leap (and the transition to 15.3 hasn't happened yet)
over all i think we shoudl consider raising or min tested/supported
python version to python 3.8 for yoga and consider updating from centos 8 to centos 9.
Dropping CentOS 8 (and thus RHEL 8) support before RHEL 9 is released seems extremely premature to me. I understand that with CentOS Stream the world got some access to in-progress RHEL versions, but it does not change the fact that RHEL 9 does not exist yet, and there seems to be no public information on when it will appear.
From your analysis it seems that we should support 3.6 (CentOS/RHEL), 3.8 (Ubuntu 20.04) and 3.9 (Fedora, openSUSE, future RHEL).
Dmitry
i also think it worth including python 3.9 in the required testing versions for python projects.
several distros will be releaseing openstack xena/wallaby with python 3.9 and also yoga likely with 3.10 in some cases. if we only offically test with 3.6 and 3.8 we could have some hard probalems to solve going forward. the optional testing with unit and functional tests that some project have for python 3.9 certenly helps but actully having an integrated tempest gate job that runs on 3.9 i think is going to be increasingly important.
readring centos 9 stream, it was orginally planned of release at the end of q2. now that date has obviosly passed but i know that they are still hopeful to have a release ready in the next month or so. cloud images are avaibel at https://composes.stream.centos.org/production/latest-CentOS-Stream/compose/B... although the repos are still only partly complete https://composes.stream.centos.org/production/latest-CentOS-Stream/compose/B... and there are currently no active mirros so its still officaly not released https://www.centos.org/centos-stream/ and i don not have any ETA beyond soonTM. as such im aware we cant offically declare centos 9 stream as part of the tested runtimes at this point but it might be at least worht listing it as an experimtal runtime as i suspec there will be a desire to swtich to it during the yoga cycle for some proejct. ooo/rdo come to mind as they discussed this for xena but obviously that was not possibel since it was not released in time.
regards sean.
ps removing python 3.6 and replacing it with 3.9 instead of just adding 3.9 as an addtional runtime is more to manage our testing there isnt nessarly any python feature that i explcitly would like to be able to use that are added in 3.8 that would be enabled by making that the new min python verion that is motivating this suggestion. there are some nice thingin 3.9/3.10 such as pattern mataching that are commign but it will be some time (2-4 years)before we can actully use those as our min version so i was just being practial and suggestion we raise to 3.8 and add 3.9 as required with 3.10 as optional/experimal for those that want it.
-- Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill
On Tue, 2021-09-07 at 10:51 +0100, Sean Mooney wrote:
hi i was jsut looking at the yoga runtimes an notice that we have not really updated it with regards to minium python versions. [...]
That was introduced in July via https://review.opendev.org/799927 but should probably be considered a placeholder for now. We determine what versions of Python to target at or just prior to the start of the development cycle in order to use the latest information available to us, while avoiding changing team focus during the cycle. There's certainly still time to update it to include Python 3.9, and we do in fact already run (mostly non-voting) unit tests of many projects with 3.9. You may as well go ahead and push your proposed edit for review, it may not need much discussion. -- Jeremy Stanley
---- On Tue, 07 Sep 2021 06:07:55 -0500 Sean Mooney <smooney@redhat.com> wrote ----
resending to correct list On Tue, 2021-09-07 at 10:51 +0100, Sean Mooney wrote:
hi i was jsut looking at the yoga runtimes an notice that we have not really updated it with regards to minium python versions.
currently
the runtime defintion is as follows https://github.com/openstack/governance/blob/master/reference/runtimes/yoga....
It is the :doc:`policy <../../resolutions/20181024-python-update-process>` that each OpenStack release cycle will target the latest available version of Python; default Python runtimes on the distributions listed above; and versions used in integration tests at the start of the cycle, at least until the point when all projects have migrated to a later version.
Based on the criteria above, all Python-based projects must target and test against, at a minimum:
Python 3.6 (default in CentOS 8 Stream) Python 3.8 (latest available; default in Ubuntu 20.04)
looking at to look at https://repology.org/project/python/versions what python version are shpping by default now in many distro more and more of them are moving to python 3.9 which is currently not in the test list and wer are starting to fall quite far behind
debian 11/12/stable are currently using 3.9.2 centos 8 defaults to 3.6.8 but ships 3.8.6 and 3.9.2 as appstreams centos 9 stream will default to python3-3.9.6-6.el9.x86_64.rpm https://composes.stream.centos.org/test/latest-CentOS-Stream/compose/BaseOS/... ubuntu 20.04 default to 3.8.2 but also ships 3.9.5 in universe ubuntu 21.04 default to 3.9.5 so we can expect that when yoga ships on ubuntu 22.04 it will be based on at least python 3.9 and proably 3.10 fedora well they basically ship everythin but i belive 3.9 is not the default based on complaints some of my fellow contiutors had with runnign unit and functional tests.
openSUSE Leap 15.3 main/oss python39 3.9.4 ship 3.9 but default to 3.6 they aslo nolonger have an openstack product but it would still be nice to keep them in mind.
Definitely we can move to 3.9 as official testing runtime if any of the currently supporting distro support it as default. openSuse distro is not supported anymore due to no/less volunteer to maintain it. centos9 or Ubuntu 22.04 LTS is not yet released so we have not picked them yet but that is something we discussed during the review of defining Yoga testing runtime. If any of these distro is released before Yoga development cycle we can propose a update to the current testing runtime - https://review.opendev.org/c/openstack/governance/+/799927/1/reference/runti... Also one thing to make a note here, py3.9 is being tested as non-voting in all the project unit tests and hopefully projects has made it green. There is no restriction to run unit or functional tests on py3.9 as voting testing. Testing runtime is just a minimum testing we except projects to do and it can be extended at any distro/versions at project level. -gmann
over all i think we shoudl consider raising or min tested/supported python version to python 3.8 for yoga and consider updating from centos 8 to centos 9. i also think it worth including python 3.9 in the required testing versions for python projects.
several distros will be releaseing openstack xena/wallaby with python 3.9 and also yoga likely with 3.10 in some cases. if we only offically test with 3.6 and 3.8 we could have some hard probalems to solve going forward. the optional testing with unit and functional tests that some project have for python 3.9 certenly helps but actully having an integrated tempest gate job that runs on 3.9 i think is going to be increasingly important.
readring centos 9 stream, it was orginally planned of release at the end of q2. now that date has obviosly passed but i know that they are still hopeful to have a release ready in the next month or so. cloud images are avaibel at https://composes.stream.centos.org/production/latest-CentOS-Stream/compose/B... although the repos are still only partly complete https://composes.stream.centos.org/production/latest-CentOS-Stream/compose/B... and there are currently no active mirros so its still officaly not released https://www.centos.org/centos-stream/ and i don not have any ETA beyond soonTM. as such im aware we cant offically declare centos 9 stream as part of the tested runtimes at this point but it might be at least worht listing it as an experimtal runtime as i suspec there will be a desire to swtich to it during the yoga cycle for some proejct. ooo/rdo come to mind as they discussed this for xena but obviously that was not possibel since it was not released in time.
regards sean.
ps removing python 3.6 and replacing it with 3.9 instead of just adding 3.9 as an addtional runtime is more to manage our testing there isnt nessarly any python feature that i explcitly would like to be able to use that are added in 3.8 that would be enabled by making that the new min python verion that is motivating this suggestion. there are some nice thingin 3.9/3.10 such as pattern mataching that are commign but it will be some time (2-4 years)before we can actully use those as our min version so i was just being practial and suggestion we raise to 3.8 and add 3.9 as required with 3.10 as optional/experimal for those that want it.
On 2021-09-07 10:38:12 -0500 (-0500), Ghanshyam Mann wrote: [...]
Definitely we can move to 3.9 as official testing runtime if any of the currently supporting distro support it as default. [...]
I don't think it needs to be the default Python in a release so long as it's available. To quote from https://governance.openstack.org/tc/resolutions/20181024-python-update-proce... "Prior to the beginning of each release cycle, the TC will designate the minor versions of Python 3 that will be tested in that release using the following criteria: [...] The latest released version of Python 3 that is available in any distribution we can feasibly use for testing. It need not be a long-term-supported release, but could be a non-LTS version of Ubuntu, Fedora, openSUSE Leap, or even a rolling release distribution (such as Debian Testing or openSUSE Tumbleweed) if necessary." I recall clearly that this was added so that we would attempt to test the latest available version of Python in order to help avoid having software which is clearly non-functional on distributions which are working in parallel to update their defaults (and so might reasonably want to ship OpenStack with an interpreter version which was not their default at the time we started development for that coordinated release). -- Jeremy Stanley
---- On Tue, 07 Sep 2021 11:03:09 -0500 Jeremy Stanley <fungi@yuggoth.org> wrote ----
On 2021-09-07 10:38:12 -0500 (-0500), Ghanshyam Mann wrote: [...]
Definitely we can move to 3.9 as official testing runtime if any of the currently supporting distro support it as default. [...]
I don't think it needs to be the default Python in a release so long as it's available. To quote from https://governance.openstack.org/tc/resolutions/20181024-python-update-proce...
"Prior to the beginning of each release cycle, the TC will designate the minor versions of Python 3 that will be tested in that release using the following criteria: [...] The latest released version of Python 3 that is available in any distribution we can feasibly use for testing. It need not be a long-term-supported release, but could be a non-LTS version of Ubuntu, Fedora, openSUSE Leap, or even a rolling release distribution (such as Debian Testing or openSUSE Tumbleweed) if necessary."
I recall clearly that this was added so that we would attempt to test the latest available version of Python in order to help avoid having software which is clearly non-functional on distributions which are working in parallel to update their defaults (and so might reasonably want to ship OpenStack with an interpreter version which was not their default at the time we started development for that coordinated release).
Sure, and by seeing the current py39 job it is mostly passing - https://zuul.openstack.org/builds?job_name=openstack-tox-py39 Let's discuss it in PTG and hopefully by then we might see centos9 release or closure plan to release dates and as discussed during defining the Yoga tetsing runtime, we can update it once if centos9 is there then update centos versionb as well as the py version otherwise we can disucss only making py39 jobs voting which are running on ubuntu focal. -gmann
-- Jeremy Stanley
On Tue, 2021-09-07 at 11:44 -0500, Ghanshyam Mann wrote:
---- On Tue, 07 Sep 2021 11:03:09 -0500 Jeremy Stanley <fungi@yuggoth.org> wrote ----
On 2021-09-07 10:38:12 -0500 (-0500), Ghanshyam Mann wrote: [...]
Definitely we can move to 3.9 as official testing runtime if any of the currently supporting distro support it as default. [...]
I don't think it needs to be the default Python in a release so long as it's available. To quote from https://governance.openstack.org/tc/resolutions/20181024-python-update-proce...
"Prior to the beginning of each release cycle, the TC will designate the minor versions of Python 3 that will be tested in that release using the following criteria: [...] The latest released version of Python 3 that is available in any distribution we can feasibly use for testing. It need not be a long-term-supported release, but could be a non-LTS version of Ubuntu, Fedora, openSUSE Leap, or even a rolling release distribution (such as Debian Testing or openSUSE Tumbleweed) if necessary."
I recall clearly that this was added so that we would attempt to test the latest available version of Python in order to help avoid having software which is clearly non-functional on distributions which are working in parallel to update their defaults (and so might reasonably want to ship OpenStack with an interpreter version which was not their default at the time we started development for that coordinated release).
Sure, and by seeing the current py39 job it is mostly passing - https://zuul.openstack.org/builds?job_name=openstack-tox-py39
Let's discuss it in PTG and hopefully by then we might see centos9 release or closure plan to release dates and as discussed during defining the Yoga tetsing runtime, we can update it once if centos9 is there then update centos versionb as well as the py version otherwise we can disucss only making py39 jobs voting which are running on ubuntu focal. not sure if centos 9 will be avaiable by ptg but i hope so. in anycaes ubuntu 21.04 default to python 3.9 and debian 11 "bullseye" which is the latest __stable__ debian release also default to python 3.9 as does fedora 34 but that a much bigger moving target.
in general 3.9 is avaiable if not default in several distro which is why i would like to include it for required testing e.g. move the current non voting jobs to voting for unit and functest at least. it woudl be nice if we could have at least on tempst job on py39 too but sure lets pick this up at the ptg. im not agaisnt keeping support for phython 3.6 and centos 8 stream for yoga but i would like to be abel to do at least some smoke tests with centos 9 stream if its avaiable. the none stream version fo centos 8 i think is end of life at the end of this year? but the centos 8 stream release will continue untile rhel 8 is eol so that shoudl still be supproted to some degree for the full yoga cycle. the none stream version however wont be supported when yoga releases.
-gmann
-- Jeremy Stanley
Sorry for top posting but I don't have a precise quote to reply to. What I wish to say is that I am trying to get Debian tested in CI but it looks like python3.9 might be playing tricks. I would appreciate it if someone more knowledgeable in Nova and Neutron internals could take a look at failures in https://review.opendev.org/c/openstack/devstack/+/789083 Do note slightly different tests fail each run. -yoctozepto On Tue, Sep 7, 2021 at 7:40 PM Sean Mooney <smooney@redhat.com> wrote:
On Tue, 2021-09-07 at 11:44 -0500, Ghanshyam Mann wrote:
---- On Tue, 07 Sep 2021 11:03:09 -0500 Jeremy Stanley <fungi@yuggoth.org> wrote ----
On 2021-09-07 10:38:12 -0500 (-0500), Ghanshyam Mann wrote: [...]
Definitely we can move to 3.9 as official testing runtime if any of the currently supporting distro support it as default. [...]
I don't think it needs to be the default Python in a release so long as it's available. To quote from https://governance.openstack.org/tc/resolutions/20181024-python-update-proce...
"Prior to the beginning of each release cycle, the TC will designate the minor versions of Python 3 that will be tested in that release using the following criteria: [...] The latest released version of Python 3 that is available in any distribution we can feasibly use for testing. It need not be a long-term-supported release, but could be a non-LTS version of Ubuntu, Fedora, openSUSE Leap, or even a rolling release distribution (such as Debian Testing or openSUSE Tumbleweed) if necessary."
I recall clearly that this was added so that we would attempt to test the latest available version of Python in order to help avoid having software which is clearly non-functional on distributions which are working in parallel to update their defaults (and so might reasonably want to ship OpenStack with an interpreter version which was not their default at the time we started development for that coordinated release).
Sure, and by seeing the current py39 job it is mostly passing - https://zuul.openstack.org/builds?job_name=openstack-tox-py39
Let's discuss it in PTG and hopefully by then we might see centos9 release or closure plan to release dates and as discussed during defining the Yoga tetsing runtime, we can update it once if centos9 is there then update centos versionb as well as the py version otherwise we can disucss only making py39 jobs voting which are running on ubuntu focal. not sure if centos 9 will be avaiable by ptg but i hope so. in anycaes ubuntu 21.04 default to python 3.9 and debian 11 "bullseye" which is the latest __stable__ debian release also default to python 3.9 as does fedora 34 but that a much bigger moving target.
in general 3.9 is avaiable if not default in several distro which is why i would like to include it for required testing e.g. move the current non voting jobs to voting for unit and functest at least. it woudl be nice if we could have at least on tempst job on py39 too but sure lets pick this up at the ptg.
im not agaisnt keeping support for phython 3.6 and centos 8 stream for yoga but i would like to be abel to do at least some smoke tests with centos 9 stream if its avaiable.
the none stream version fo centos 8 i think is end of life at the end of this year? but the centos 8 stream release will continue untile rhel 8 is eol so that shoudl still be supproted to some degree for the full yoga cycle. the none stream version however wont be supported when yoga releases.
-gmann
-- Jeremy Stanley
participants (5)
-
Dmitry Tantsur
-
Ghanshyam Mann
-
Jeremy Stanley
-
Radosław Piliszek
-
Sean Mooney