[all][tc] Planning for dropping the Python2 support in OpenStack
Hello Everyone,
Python 2.7 is going to retire in Jan 2020 [1] and we planned to drop the python 2 support from OpenStack during the start of the Ussuri cycle[2].
Time has come now to start the planning on dropping the Python2. It needs to be coordinated among various Projects, libraries, vendors driver, third party CI and testing frameworks.
* Preparation for the Plan & Schedule:
Etherpad: https://etherpad.openstack.org/p/drop-python2-support
We discussed it in TC to come up with the plan, execute it smoothly and avoid breaking any dependent projects. I have prepared an etherpad[3](mentioned above also) to capture all the points related to this topic and most importantly the draft schedule about who can drop the support and when. The schedule is in the draft state and not final yet. The most important points are if you are dropping the support then all your consumers (OpenStack Projects, Vendors drivers etc) are ready for that. For example, oslo, os-bricks, client lib, testing framework projects will keep the python2 support until we make sure all the consumers of those projects do not require py2 support. If anyone require then how long they can support py2. These libraries, testing frameworks will be the last one to drop py2.
We have planned to have a dedicated discussion in TC office hours on the 24th Thursday #openstack-tc channel. We will discuss what all need to be done and the schedules.
You do not have to drop it immediately and keep eyes on this ML thread till we get the consensus on the community-level plan and schedule.
Meanwhile, you can always start pre-planning for your projects, for example, stephenfin has started for Nova[4] to migrate the third party CI etc. Cinder has coordinated with all vendor drivers & their CI to migrate from py2 to py3.
* Projects want to keep the py2 support? There is no mandate that projects have to drop the py2 support right now. If you want to keep the support then key things to discuss are what all you need and does all your dependent projects/libs provide the support of py2. This is something needs to be discussed case by case. If any project wants to keep the support, add that in the etherpad with a brief reason which will be helpful to discuss the need and feasibility.
Feel free to provide feedback or add the missing point on the etherpad. Do not forget to attend the 24th Oct 2019, TC office hour on Thursday at 1500 UTC in #openstack-tc.
[1] https://pythonclock.org/ [2] https://governance.openstack.org/tc/resolutions/20180529-python2-deprecation... [3] https://etherpad.openstack.org/p/drop-python2-support [4] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010109.h...
-gmann
Just a reminder, discussion for this is going to start in #openstack-tc channel in another 5 min.
-gmann
---- On Tue, 15 Oct 2019 13:18:03 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Hello Everyone,
Python 2.7 is going to retire in Jan 2020 [1] and we planned to drop the python 2 support from OpenStack during the start of the Ussuri cycle[2].
Time has come now to start the planning on dropping the Python2. It needs to be coordinated among various Projects, libraries, vendors driver, third party CI and testing frameworks.
- Preparation for the Plan & Schedule:
Etherpad: https://etherpad.openstack.org/p/drop-python2-support
We discussed it in TC to come up with the plan, execute it smoothly and avoid breaking any dependent projects. I have prepared an etherpad[3](mentioned above also) to capture all the points related to this topic and most importantly the draft schedule about who can drop the support and when. The schedule is in the draft state and not final yet. The most important points are if you are dropping the support then all your consumers (OpenStack Projects, Vendors drivers etc) are ready for that. For example, oslo, os-bricks, client lib, testing framework projects will keep the python2 support until we make sure all the consumers of those projects do not require py2 support. If anyone require then how long they can support py2. These libraries, testing frameworks will be the last one to drop py2.
We have planned to have a dedicated discussion in TC office hours on the 24th Thursday #openstack-tc channel. We will discuss what all need to be done and the schedules.
You do not have to drop it immediately and keep eyes on this ML thread till we get the consensus on the community-level plan and schedule.
Meanwhile, you can always start pre-planning for your projects, for example, stephenfin has started for Nova[4] to migrate the third party CI etc. Cinder has coordinated with all vendor drivers & their CI to migrate from py2 to py3.
- Projects want to keep the py2 support?
There is no mandate that projects have to drop the py2 support right now. If you want to keep the support then key things to discuss are what all you need and does all your dependent projects/libs provide the support of py2. This is something needs to be discussed case by case. If any project wants to keep the support, add that in the etherpad with a brief reason which will be helpful to discuss the need and feasibility.
Feel free to provide feedback or add the missing point on the etherpad. Do not forget to attend the 24th Oct 2019, TC office hour on Thursday at 1500 UTC in #openstack-tc.
[1] https://pythonclock.org/ [2] https://governance.openstack.org/tc/resolutions/20180529-python2-deprecation... [3] https://etherpad.openstack.org/p/drop-python2-support [4] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010109.h...
-gmann
Hello Everyone,
We had good amount of discussion on the final plan and schedule in today's TC office hour[1].
I captured the agreement on each point in etherpad (you can see the AGREE:). Also summarizing the discussions here. Imp point is if your projects are planning to keep the py2.7 support then do not delay to tell us. Reply on this ML thread or add your project in etherpad.
- Projects can start dropping the py2.7 support. Common lib and testing tools need to wait until milestone-2. ** pepe8 job to be included in openstack-python3-ussuri-jobs-* templates - https://review.opendev.org/#/c/688997/ ** You can drop openstack-python-jobs template and start using ussuri template once 688997 patch is merged. ** Cross projects dependency (if any ) can be sync up among dependent projects.
- I will add this plan and schedule as a community goal. The goal is more about what all things to do and when. ** If any project keeping the support then it has to be notified explicitly for its consumer.
- Schedule: The schedule is aligned with the Ussuri cycle milestone[2]. I will add the plan in the release schedule also. Phase-1: Dec 09 - Dec 13 R-22 Ussuri-1 milestone ** Project to start dropping the py2 support along with all the py2 CI jobs. Phase-2: Feb 10 - Feb 14 R-13 Ussuri-2 milestone ** This includes Oslo, QA tools (or any other testing tools), common lib (os-brick), Client library. ** This will give enough time to projects to drop the py2 support. Phase-3: Apr 06 - Apr 10 R-5 Ussuri-3 milestone ** Final audit on Phase-1 and Phase-2 plan and make sure everything is done without breaking anything. This is enough time to measure such break or anything extra to do before ussuri final release.
Other discussions points and agreement: - Projects want to keep python 2 support and need oslo, QA or any other dependent projects/lib support: ** swift. AI: gmann to reach out to swift team about the plan and exact required things from its dependency (the common lib/testing tool). ** List your project if you want to keep the py2 support. ** Action item: TC liaisons to reach out to their projects and make sure they are aware of this plan[3].
- How to test the upgrade from python2 -> python3 ** AGREE: let's drop the integrated testing for py2->py3 and oslo can check if they can do functional testing of oslo.messaging - https://bugs.launchpad.net/oslo.messaging/+bug/1792977
- What are our guidelines to backport fixes to stable branches? ** AGREE: This will be rare case and if that happen then tweaking for py27 in the backport is fine. The stable branch backport policy does not need any changing for this
- What is the tactics of dropping openstack-tox-py27 in our gate? ** AGREE: on merging the pep8 job in ussuri job template https://review.opendev.org/#/c/688997/
[1] http://eavesdrop.openstack.org/irclogs/%23openstack-tc/%23openstack-tc.2019-... [2] https://releases.openstack.org/ussuri/schedule.html [3] https://governance.openstack.org/tc/reference/tc-liaisons.html
-gmann
---- On Thu, 24 Oct 2019 09:55:58 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Just a reminder, discussion for this is going to start in #openstack-tc channel in another 5 min.
-gmann
---- On Tue, 15 Oct 2019 13:18:03 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Hello Everyone,
Python 2.7 is going to retire in Jan 2020 [1] and we planned to drop the python 2 support from OpenStack during the start of the Ussuri cycle[2].
Time has come now to start the planning on dropping the Python2. It needs to be coordinated among various Projects, libraries, vendors driver, third party CI and testing frameworks.
- Preparation for the Plan & Schedule:
Etherpad: https://etherpad.openstack.org/p/drop-python2-support
We discussed it in TC to come up with the plan, execute it smoothly and avoid breaking any dependent projects. I have prepared an etherpad[3](mentioned above also) to capture all the points related to this topic and most importantly the draft schedule about who can drop the support and when. The schedule is in the draft state and not final yet. The most important points are if you are dropping the support then all your consumers (OpenStack Projects, Vendors drivers etc) are ready for that. For example, oslo, os-bricks, client lib, testing framework projects will keep the python2 support until we make sure all the consumers of those projects do not require py2 support. If anyone require then how long they can support py2. These libraries, testing frameworks will be the last one to drop py2.
We have planned to have a dedicated discussion in TC office hours on the 24th Thursday #openstack-tc channel. We will discuss what all need to be done and the schedules.
You do not have to drop it immediately and keep eyes on this ML thread till we get the consensus on the community-level plan and schedule.
Meanwhile, you can always start pre-planning for your projects, for example, stephenfin has started for Nova[4] to migrate the third party CI etc. Cinder has coordinated with all vendor drivers & their CI to migrate from py2 to py3.
- Projects want to keep the py2 support?
There is no mandate that projects have to drop the py2 support right now. If you want to keep the support then key things to discuss are what all you need and does all your dependent projects/libs provide the support of py2. This is something needs to be discussed case by case. If any project wants to keep the support, add that in the etherpad with a brief reason which will be helpful to discuss the need and feasibility.
Feel free to provide feedback or add the missing point on the etherpad. Do not forget to attend the 24th Oct 2019, TC office hour on Thursday at 1500 UTC in #openstack-tc.
[1] https://pythonclock.org/ [2] https://governance.openstack.org/tc/resolutions/20180529-python2-deprecation... [3] https://etherpad.openstack.org/p/drop-python2-support [4] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010109.h...
-gmann
On 2019-10-24 14:32:03 -0500 (-0500), Ghanshyam Mann wrote: [...]
- Projects can start dropping the py2.7 support. Common lib and
testing tools need to wait until milestone-2.
[...]
This doesn't match the intent behind what I originally suggested nor my subsequent interpretation of what we discussed, and unfortunately the plan on the etherpad is slightly vague here too. I thought what we were agreeing to was that leaf projects (services and the like) had *until* milestone 1 (~2019-12-12) to remove Python 2.7 testing if they depend on shared libraries which are planning to remove support for it in Ussuri. From then until milestone 2 (~2020-02-13) shared libraries could work on dropping support for Python 2.7. If libs are allowed to drop support for it *after* milestone 2 then that doesn't leave much time before they're released at milestone 3 to stabilize or reverse course.
Phase-1: Dec 09 - Dec 13 R-22 Ussuri-1 milestone ** Project to start dropping the py2 support along with all the py2 CI jobs.
This is a milestone later than I expected, unless you mean they should be done by this point. It's just about removing jobs, so projects should be on the ball and do this quickly.
Phase-2: Feb 10 - Feb 14 R-13 Ussuri-2 milestone ** This includes Oslo, QA tools (or any other testing tools), common lib (os-brick), Client library. ** This will give enough time to projects to drop the py2 support.
This leaves less than 2 months where libraries are allowed to complete the necessary work before they get released for Ussuri (remember the final release for libraries is at R-6, the week before milestone 3).
Phase-3: Apr 06 - Apr 10 R-5 Ussuri-3 milestone ** Final audit on Phase-1 and Phase-2 plan and make sure everything is done without breaking anything. This is enough time to measure such break or anything extra to do before ussuri final release.
[...]
Libraries are released the week before this, so no, that doesn't really provide any auditing opportunity.
I apologize, in retrospect I realize that the distinction between "by" and "at" in English could be too subtle for a lot of folks to pick up on, and I should have been more explicit in my original proposal.
---- On Thu, 24 Oct 2019 16:57:55 -0500 Jeremy Stanley fungi@yuggoth.org wrote ----
On 2019-10-24 14:32:03 -0500 (-0500), Ghanshyam Mann wrote: [...]
- Projects can start dropping the py2.7 support. Common lib and
testing tools need to wait until milestone-2.
[...]
This doesn't match the intent behind what I originally suggested nor my subsequent interpretation of what we discussed, and unfortunately the plan on the etherpad is slightly vague here too. I thought what we were agreeing to was that leaf projects (services and the like) had *until* milestone 1 (~2019-12-12) to remove Python 2.7 testing if they depend on shared libraries which are planning to remove support for it in Ussuri. From then until milestone 2 (~2020-02-13) shared libraries could work on dropping support for Python 2.7. If libs are allowed to drop support for it *after* milestone 2 then that doesn't leave much time before they're released at milestone 3 to stabilize or reverse course.
Phase-1: Dec 09 - Dec 13 R-22 Ussuri-1 milestone ** Project to start dropping the py2 support along with all the py2 CI jobs.
This is a milestone later than I expected, unless you mean they should be done by this point. It's just about removing jobs, so projects should be on the ball and do this quickly.
Phase-2: Feb 10 - Feb 14 R-13 Ussuri-2 milestone ** This includes Oslo, QA tools (or any other testing tools), common lib (os-brick), Client library. ** This will give enough time to projects to drop the py2 support.
This leaves less than 2 months where libraries are allowed to complete the necessary work before they get released for Ussuri (remember the final release for libraries is at R-6, the week before milestone 3).
Phase-3: Apr 06 - Apr 10 R-5 Ussuri-3 milestone ** Final audit on Phase-1 and Phase-2 plan and make sure everything is done without breaking anything. This is enough time to measure such break or anything extra to do before ussuri final release.
[...]
Libraries are released the week before this, so no, that doesn't really provide any auditing opportunity.
Sorry for the confusion in the schedule. Below one is what I meant.
Phase-1: Now -> Ussuri-1 milestone (deadline R-22 ) ** Project to dropping the py2 support along with all the py2 CI jobs.
Phase-2: milestone-1 -> milestone-2 ( deadline R-13 ) ** This includes Oslo, QA tools (or any other testing tools), common lib (os-brick), Client library.
Phase-3: at milestone-2 ** Final audit on Phase-1 and Phase-2 plan and make sure everything is done without breaking anything. This is enough time to measure such a break or anything extra to do before Ussuri final release.
-gmann
I apologize, in retrospect I realize that the distinction between "by" and "at" in English could be too subtle for a lot of folks to pick up on, and I should have been more explicit in my original proposal. -- Jeremy Stanley
On Thu, Oct 24, 2019 at 3:58 PM Ghanshyam Mann gmann@ghanshyammann.com wrote:
---- On Thu, 24 Oct 2019 16:57:55 -0500 Jeremy Stanley fungi@yuggoth.org wrote ----
On 2019-10-24 14:32:03 -0500 (-0500), Ghanshyam Mann wrote: [...]
- Projects can start dropping the py2.7 support. Common lib and
testing tools need to wait until milestone-2.
[...]
This doesn't match the intent behind what I originally suggested nor my subsequent interpretation of what we discussed, and unfortunately the plan on the etherpad is slightly vague here too. I thought what we were agreeing to was that leaf projects (services and the like) had *until* milestone 1 (~2019-12-12) to remove Python 2.7 testing if they depend on shared libraries which are planning to remove support for it in Ussuri. From then until milestone 2 (~2020-02-13) shared libraries could work on dropping support for Python 2.7. If libs are allowed to drop support for it *after* milestone 2 then that doesn't leave much time before they're released at milestone 3 to stabilize or reverse course.
Phase-1: Dec 09 - Dec 13 R-22 Ussuri-1 milestone ** Project to start dropping the py2 support along with all the py2 CI jobs.
This is a milestone later than I expected, unless you mean they should be done by this point. It's just about removing jobs, so projects should be on the ball and do this quickly.
Phase-2: Feb 10 - Feb 14 R-13 Ussuri-2 milestone ** This includes Oslo, QA tools (or any other testing tools), common lib (os-brick), Client library. ** This will give enough time to projects to drop the py2 support.
This leaves less than 2 months where libraries are allowed to complete the necessary work before they get released for Ussuri (remember the final release for libraries is at R-6, the week before milestone 3).
Phase-3: Apr 06 - Apr 10 R-5 Ussuri-3 milestone ** Final audit on Phase-1 and Phase-2 plan and make sure everything is done without breaking anything. This is enough time to measure such break or anything extra to do before ussuri final release.
[...]
Libraries are released the week before this, so no, that doesn't really provide any auditing opportunity.
Sorry for the confusion in the schedule. Below one is what I meant.
Phase-1: Now -> Ussuri-1 milestone (deadline R-22 ) ** Project to dropping the py2 support along with all the py2 CI jobs.
Phase-2: milestone-1 -> milestone-2 ( deadline R-13 ) ** This includes Oslo, QA tools (or any other testing tools), common lib (os-brick), Client library.
Phase-3: at milestone-2 ** Final audit on Phase-1 and Phase-2 plan and make sure everything is done without breaking anything. This is enough time to measure such a break or anything extra to do before Ussuri final release.
Awesome, thank you for stating this very clearly. For OpenStack manila, I've lined up the patches like you've indicated:
Phase-1: Now -> Ussuri-1 milestone (deadline R-22 ) ** openstack/manila will drop support for python2.7: https://review.opendev.org/#/c/691134/
Phase-2: milestone-1 -> milestone-2 ( deadline R-13 ) ** Client projects and tempest plugin will drop support for python2.7: https://review.opendev.org/#/c/691183/ (openstack/python-manilaclient) https://review.opendev.org/#/c/691186/ (openstack/manila-tempest-plugin) https://review.opendev.org/#/c/691184/ (openstack/manila-ui)
Phase-3: at milestone-2 ** Final audit on Phase-1 and Phase-2 plan and make sure everything is done without breaking anything.
I'm using the Gerrit topic "drop-py2" to track these.
-gmann
I apologize, in retrospect I realize that the distinction between "by" and "at" in English could be too subtle for a lot of folks to pick up on, and I should have been more explicit in my original proposal. -- Jeremy Stanley
---- On Fri, 25 Oct 2019 01:38:32 -0500 Goutham Pacha Ravi gouthampravi@gmail.com wrote ----
On Thu, Oct 24, 2019 at 3:58 PM Ghanshyam Mann gmann@ghanshyammann.com wrote: ---- On Thu, 24 Oct 2019 16:57:55 -0500 Jeremy Stanley fungi@yuggoth.org wrote ----
On 2019-10-24 14:32:03 -0500 (-0500), Ghanshyam Mann wrote: [...]
- Projects can start dropping the py2.7 support. Common lib and
testing tools need to wait until milestone-2.
[...]
This doesn't match the intent behind what I originally suggested nor my subsequent interpretation of what we discussed, and unfortunately the plan on the etherpad is slightly vague here too. I thought what we were agreeing to was that leaf projects (services and the like) had *until* milestone 1 (~2019-12-12) to remove Python 2.7 testing if they depend on shared libraries which are planning to remove support for it in Ussuri. From then until milestone 2 (~2020-02-13) shared libraries could work on dropping support for Python 2.7. If libs are allowed to drop support for it *after* milestone 2 then that doesn't leave much time before they're released at milestone 3 to stabilize or reverse course.
Phase-1: Dec 09 - Dec 13 R-22 Ussuri-1 milestone ** Project to start dropping the py2 support along with all the py2 CI jobs.
This is a milestone later than I expected, unless you mean they should be done by this point. It's just about removing jobs, so projects should be on the ball and do this quickly.
Phase-2: Feb 10 - Feb 14 R-13 Ussuri-2 milestone ** This includes Oslo, QA tools (or any other testing tools), common lib (os-brick), Client library. ** This will give enough time to projects to drop the py2 support.
This leaves less than 2 months where libraries are allowed to complete the necessary work before they get released for Ussuri (remember the final release for libraries is at R-6, the week before milestone 3).
Phase-3: Apr 06 - Apr 10 R-5 Ussuri-3 milestone ** Final audit on Phase-1 and Phase-2 plan and make sure everything is done without breaking anything. This is enough time to measure such break or anything extra to do before ussuri final release.
[...]
Libraries are released the week before this, so no, that doesn't really provide any auditing opportunity.
Sorry for the confusion in the schedule. Below one is what I meant.
Phase-1: Now -> Ussuri-1 milestone (deadline R-22 ) ** Project to dropping the py2 support along with all the py2 CI jobs.
Phase-2: milestone-1 -> milestone-2 ( deadline R-13 ) ** This includes Oslo, QA tools (or any other testing tools), common lib (os-brick), Client library.
Phase-3: at milestone-2 ** Final audit on Phase-1 and Phase-2 plan and make sure everything is done without breaking anything. This is enough time to measure such a break or anything extra to do before Ussuri final release.
Awesome, thank you for stating this very clearly. For OpenStack manila, I've lined up the patches like you've indicated: Phase-1: Now -> Ussuri-1 milestone (deadline R-22 )** openstack/manila will drop support for python2.7: https://review.opendev.org/#/c/691134/ Phase-2: milestone-1 -> milestone-2 ( deadline R-13 )** Client projects and tempest plugin will drop support for python2.7: https://review.opendev.org/#/c/691183/ (openstack/python-manilaclient) https://review.opendev.org/#/c/691186/ (openstack/manila-tempest-plugin) https://review.opendev.org/#/c/691184/ (openstack/manila-ui) Phase-3: at milestone-2** Final audit on Phase-1 and Phase-2 plan and make sure everything is done without breaking anything. I'm using the Gerrit topic "drop-py2" to track these.
Perfect things Gautham.
I have drafted this as a goal also - https://review.opendev.org/#/c/691178/
-gmann
-gmann
I apologize, in retrospect I realize that the distinction between "by" and "at" in English could be too subtle for a lot of folks to pick up on, and I should have been more explicit in my original proposal. -- Jeremy Stanley
---- On Thu, 24 Oct 2019 14:32:03 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Hello Everyone,
We had good amount of discussion on the final plan and schedule in today's TC office hour[1].
I captured the agreement on each point in etherpad (you can see the AGREE:). Also summarizing the discussions here. Imp point is if your projects are planning to keep the py2.7 support then do not delay to tell us. Reply on this ML thread or add your project in etherpad.
Projects can start dropping the py2.7 support. Common lib and testing tools need to wait until milestone-2. ** pepe8 job to be included in openstack-python3-ussuri-jobs-* templates - https://review.opendev.org/#/c/688997/ ** You can drop openstack-python-jobs template and start using ussuri template once 688997 patch is merged. ** Cross projects dependency (if any ) can be sync up among dependent projects.
I will add this plan and schedule as a community goal. The goal is more about what all things to do and when. ** If any project keeping the support then it has to be notified explicitly for its consumer.
Schedule:
The schedule is aligned with the Ussuri cycle milestone[2]. I will add the plan in the release schedule also. Phase-1: Dec 09 - Dec 13 R-22 Ussuri-1 milestone ** Project to start dropping the py2 support along with all the py2 CI jobs. Phase-2: Feb 10 - Feb 14 R-13 Ussuri-2 milestone ** This includes Oslo, QA tools (or any other testing tools), common lib (os-brick), Client library. ** This will give enough time to projects to drop the py2 support. Phase-3: Apr 06 - Apr 10 R-5 Ussuri-3 milestone ** Final audit on Phase-1 and Phase-2 plan and make sure everything is done without breaking anything. This is enough time to measure such break or anything extra to do before ussuri final release.
Other discussions points and agreement:
- Projects want to keep python 2 support and need oslo, QA or any other dependent projects/lib support: ** swift. AI: gmann to reach out to swift team about the plan and exact required things from its dependency (the common lib/testing tool).
I chated with timburke on IRC about things required by swift to keep the py2.7 support[1]. Below are client lib/middleware swift required for py2 testing. @timburke, feel free to update if any missing point.
- devstack. able to keep running swift on py2 and rest all services can be on py3 - keystonemiddleware and its dependency - keystoneclient and openstackclient (dep of keystonemiddleware) - castellan and barbicanclient
As those lib/middleware going to drop the py2.7 support in phase-2, we need to cap them for swift. I think capping them for python2.7 in upper constraint file would not affect any other users but Matthew Thode can explain better how that will work from the requirement constraint perspective.
[1] http://eavesdrop.openstack.org/irclogs/%23openstack-swift/%23openstack-swift...
-gmann
** List your project if you want to keep the py2 support. ** Action item: TC liaisons to reach out to their projects and make sure they are aware of this plan[3].
How to test the upgrade from python2 -> python3 ** AGREE: let's drop the integrated testing for py2->py3 and oslo can check if they can do functional testing of oslo.messaging - https://bugs.launchpad.net/oslo.messaging/+bug/1792977
What are our guidelines to backport fixes to stable branches? ** AGREE: This will be rare case and if that happen then tweaking for py27 in the backport is fine. The stable branch backport policy does not need any changing for this
What is the tactics of dropping openstack-tox-py27 in our gate? ** AGREE: on merging the pep8 job in ussuri job template https://review.opendev.org/#/c/688997/
[1] http://eavesdrop.openstack.org/irclogs/%23openstack-tc/%23openstack-tc.2019-... [2] https://releases.openstack.org/ussuri/schedule.html [3] https://governance.openstack.org/tc/reference/tc-liaisons.html
-gmann
---- On Thu, 24 Oct 2019 09:55:58 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Just a reminder, discussion for this is going to start in #openstack-tc channel in another 5 min.
-gmann
---- On Tue, 15 Oct 2019 13:18:03 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Hello Everyone,
Python 2.7 is going to retire in Jan 2020 [1] and we planned to drop the python 2 support from OpenStack during the start of the Ussuri cycle[2].
Time has come now to start the planning on dropping the Python2. It needs to be coordinated among various Projects, libraries, vendors driver, third party CI and testing frameworks.
- Preparation for the Plan & Schedule:
Etherpad: https://etherpad.openstack.org/p/drop-python2-support
We discussed it in TC to come up with the plan, execute it smoothly and avoid breaking any dependent projects. I have prepared an etherpad[3](mentioned above also) to capture all the points related to this topic and most importantly the draft schedule about who can drop the support and when. The schedule is in the draft state and not final yet. The most important points are if you are dropping the support then all your consumers (OpenStack Projects, Vendors drivers etc) are ready for that. For example, oslo, os-bricks, client lib, testing framework projects will keep the python2 support until we make sure all the consumers of those projects do not require py2 support. If anyone require then how long they can support py2. These libraries, testing frameworks will be the last one to drop py2.
We have planned to have a dedicated discussion in TC office hours on the 24th Thursday #openstack-tc channel. We will discuss what all need to be done and the schedules.
You do not have to drop it immediately and keep eyes on this ML thread till we get the consensus on the community-level plan and schedule.
Meanwhile, you can always start pre-planning for your projects, for example, stephenfin has started for Nova[4] to migrate the third party CI etc. Cinder has coordinated with all vendor drivers & their CI to migrate from py2 to py3.
- Projects want to keep the py2 support?
There is no mandate that projects have to drop the py2 support right now. If you want to keep the support then key things to discuss are what all you need and does all your dependent projects/libs provide the support of py2. This is something needs to be discussed case by case. If any project wants to keep the support, add that in the etherpad with a brief reason which will be helpful to discuss the need and feasibility.
Feel free to provide feedback or add the missing point on the etherpad. Do not forget to attend the 24th Oct 2019, TC office hour on Thursday at 1500 UTC in #openstack-tc.
[1] https://pythonclock.org/ [2] https://governance.openstack.org/tc/resolutions/20180529-python2-deprecation... [3] https://etherpad.openstack.org/p/drop-python2-support [4] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010109.h...
-gmann
On 10/29/2019 2:53 PM, Ghanshyam Mann wrote:
I chated with timburke on IRC about things required by swift to keep the py2.7 support[1]. Below are client lib/middleware swift required for py2 testing. @timburke, feel free to update if any missing point.
- keystoneclient and openstackclient (dep of keystonemiddleware)
openstackclient is not a library so I'm a bit confused about the point about keeping python2 support in python-openstackclient for swift.
I mention it because Eric Fried noticed today [1] that CI is blocked for OSC because the osc-functional-devstack* jobs seem to have mysteriously switched over to python3 but the functional tox target is expecting python2. So there is talk about moving OSC forward and dropping python2 support as a result of that.
[1] https://review.opendev.org/#/c/691980/
---- On Tue, 29 Oct 2019 16:32:59 -0500 Matt Riedemann mriedemos@gmail.com wrote ----
On 10/29/2019 2:53 PM, Ghanshyam Mann wrote:
I chated with timburke on IRC about things required by swift to keep the py2.7 support[1]. Below are client lib/middleware swift required for py2 testing. @timburke, feel free to update if any missing point.
- keystoneclient and openstackclient (dep of keystonemiddleware)
openstackclient is not a library so I'm a bit confused about the point about keeping python2 support in python-openstackclient for swift.
We do not need to keep the py2 in OSC, we are good to drop it in phase-1 and other client lib (python-*client) are target in phase-2.
As swift keeping the py2.7 support, we need to find out how we can make it work for swift and its py2.7 testing. Capping the swift's dependency version for python2.7 is an option where dependency can drop the support and swift use of the older version for py2.7 support/testing.
Till now there is no strict requirement from anyone to block anyone to drop the py2.7. If any project like Swift which is the only project in the list of keeping the support has other projects/lib dependency then they need to work with version cap (as long as dependency also keeping the support).
-gmann
I mention it because Eric Fried noticed today [1] that CI is blocked for OSC because the osc-functional-devstack* jobs seem to have mysteriously switched over to python3 but the functional tox target is expecting python2. So there is talk about moving OSC forward and dropping python2 support as a result of that.
[1] https://review.opendev.org/#/c/691980/
--
Thanks,
Matt
On 19-10-29 14:53:11, Ghanshyam Mann wrote:
---- On Thu, 24 Oct 2019 14:32:03 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Hello Everyone,
We had good amount of discussion on the final plan and schedule in today's TC office hour[1].
I captured the agreement on each point in etherpad (you can see the AGREE:). Also summarizing the discussions here. Imp point is if your projects are planning to keep the py2.7 support then do not delay to tell us. Reply on this ML thread or add your project in etherpad.
Projects can start dropping the py2.7 support. Common lib and testing tools need to wait until milestone-2. ** pepe8 job to be included in openstack-python3-ussuri-jobs-* templates - https://review.opendev.org/#/c/688997/ ** You can drop openstack-python-jobs template and start using ussuri template once 688997 patch is merged. ** Cross projects dependency (if any ) can be sync up among dependent projects.
I will add this plan and schedule as a community goal. The goal is more about what all things to do and when. ** If any project keeping the support then it has to be notified explicitly for its consumer.
Schedule:
The schedule is aligned with the Ussuri cycle milestone[2]. I will add the plan in the release schedule also. Phase-1: Dec 09 - Dec 13 R-22 Ussuri-1 milestone ** Project to start dropping the py2 support along with all the py2 CI jobs. Phase-2: Feb 10 - Feb 14 R-13 Ussuri-2 milestone ** This includes Oslo, QA tools (or any other testing tools), common lib (os-brick), Client library. ** This will give enough time to projects to drop the py2 support. Phase-3: Apr 06 - Apr 10 R-5 Ussuri-3 milestone ** Final audit on Phase-1 and Phase-2 plan and make sure everything is done without breaking anything. This is enough time to measure such break or anything extra to do before ussuri final release.
Other discussions points and agreement:
- Projects want to keep python 2 support and need oslo, QA or any other dependent projects/lib support: ** swift. AI: gmann to reach out to swift team about the plan and exact required things from its dependency (the common lib/testing tool).
I chated with timburke on IRC about things required by swift to keep the py2.7 support[1]. Below are client lib/middleware swift required for py2 testing. @timburke, feel free to update if any missing point.
- devstack. able to keep running swift on py2 and rest all services can be on py3
- keystonemiddleware and its dependency
- keystoneclient and openstackclient (dep of keystonemiddleware)
- castellan and barbicanclient
As those lib/middleware going to drop the py2.7 support in phase-2, we need to cap them for swift. I think capping them for python2.7 in upper constraint file would not affect any other users but Matthew Thode can explain better how that will work from the requirement constraint perspective.
[1] http://eavesdrop.openstack.org/irclogs/%23openstack-swift/%23openstack-swift...
-gmann
ya, there are examples already for libs that have dropped py2 support. What you need to do is update global requirements to be something like the following.
sphinx!=1.6.6,!=1.6.7,<2.0.0;python_version=='2.7' # BSD sphinx!=1.6.6,!=1.6.7,!=2.1.0;python_version>='3.4' # BSD
or
keyring<19.0.0;python_version=='2.7' # MIT/PSF keyring;python_version>='3.4' # MIT/PSF
On Tue, 2019-10-29 at 19:40 -0500, Matthew Thode wrote:
On 19-10-29 14:53:11, Ghanshyam Mann wrote:
---- On Thu, 24 Oct 2019 14:32:03 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Hello Everyone,
We had good amount of discussion on the final plan and schedule in today's TC office hour[1].
I captured the agreement on each point in etherpad (you can see the AGREE:). Also summarizing the discussions here. Imp point is if your projects are planning to keep the py2.7 support then do not delay to tell us. Reply on this ML thread or add your project in etherpad.
- Projects can start dropping the py2.7 support. Common lib and testing tools need to wait until milestone-2. ** pepe8 job to be included in openstack-python3-ussuri-jobs-* templates -
https://review.opendev.org/#/c/688997/
** You can drop openstack-python-jobs template and start using ussuri template once 688997 patch is merged. ** Cross projects dependency (if any ) can be sync up among dependent projects.
I will add this plan and schedule as a community goal. The goal is more about what all things to do and when. ** If any project keeping the support then it has to be notified explicitly for its consumer.
Schedule:
The schedule is aligned with the Ussuri cycle milestone[2]. I will add the plan in the release schedule also. Phase-1: Dec 09 - Dec 13 R-22 Ussuri-1 milestone ** Project to start dropping the py2 support along with all the py2 CI jobs. Phase-2: Feb 10 - Feb 14 R-13 Ussuri-2 milestone ** This includes Oslo, QA tools (or any other testing tools), common lib (os-brick), Client library. ** This will give enough time to projects to drop the py2 support. Phase-3: Apr 06 - Apr 10 R-5 Ussuri-3 milestone ** Final audit on Phase-1 and Phase-2 plan and make sure everything is done without breaking anything. This is enough time to measure such break or anything extra to do before ussuri final release.
Other discussions points and agreement:
- Projects want to keep python 2 support and need oslo, QA or any other dependent projects/lib support: ** swift. AI: gmann to reach out to swift team about the plan and exact required things from its dependency (the common lib/testing tool).
I chated with timburke on IRC about things required by swift to keep the py2.7 support[1]. Below are client lib/middleware swift required for py2 testing. @timburke, feel free to update if any missing point.
- devstack. able to keep running swift on py2 and rest all services can be on py3
- keystonemiddleware and its dependency
- keystoneclient and openstackclient (dep of keystonemiddleware)
- castellan and barbicanclient
As those lib/middleware going to drop the py2.7 support in phase-2, we need to cap them for swift. I think capping them for python2.7 in upper constraint file would not affect any other users but Matthew Thode can explain better how that will work from the requirement constraint perspective.
[1] http://eavesdrop.openstack.org/irclogs/%23openstack-swift/%23openstack-swift...
-gmann
ya, there are examples already for libs that have dropped py2 support. What you need to do is update global requirements to be something like the following.
sphinx!=1.6.6,!=1.6.7,<2.0.0;python_version=='2.7' # BSD sphinx!=1.6.6,!=1.6.7,!=2.1.0;python_version>='3.4' # BSD
or
keyring<19.0.0;python_version=='2.7' # MIT/PSF keyring;python_version>='3.4' # MIT/PSF
on a related note os-vif is blocked form running tempest jobs under python 3 until https://review.opendev.org/#/c/681029/ is merged due to https://zuul.opendev.org/t/openstack/build/4ff60d6bd2f24782abeb12cc7bdb8013/...
i think this issue will affect any job that install proejcts that use privsep using the required-proejcts section of the zuul job definition. adding a project to required-proejcts sechtion adds it to the LIBS_FROM_GIT varible in devstack. this inturn istalls it twice due to https://review.opendev.org/#/c/418135/ . the side effect of this is that the privsep helper script gets installed under python2 and the neutron ageint in this case gets install under python 3 so when it trys to spawn the privsep deamon and invoke commands it typically expodes due to dependcy issues or in this case because it failed to drop privileges correctly.
so as part of phase 1 we need to merge https://review.opendev.org/#/c/681029/ so that lib project that use required- projects to run with master of project that comsume it and support depends-on can move to python 3 tempest jobs.
---- On Wed, 30 Oct 2019 06:59:19 -0500 Sean Mooney smooney@redhat.com wrote ----
On Tue, 2019-10-29 at 19:40 -0500, Matthew Thode wrote:
On 19-10-29 14:53:11, Ghanshyam Mann wrote:
---- On Thu, 24 Oct 2019 14:32:03 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Hello Everyone,
We had good amount of discussion on the final plan and schedule in today's TC office hour[1].
I captured the agreement on each point in etherpad (you can see the AGREE:). Also summarizing the discussions here. Imp point is if your projects are planning to keep the py2.7 support then do not delay to tell us. Reply on this ML thread or add your project in etherpad.
- Projects can start dropping the py2.7 support. Common lib and testing tools need to wait until milestone-2. ** pepe8 job to be included in openstack-python3-ussuri-jobs-* templates -
https://review.opendev.org/#/c/688997/
** You can drop openstack-python-jobs template and start using ussuri template once 688997 patch is merged. ** Cross projects dependency (if any ) can be sync up among dependent projects.
I will add this plan and schedule as a community goal. The goal is more about what all things to do and when. ** If any project keeping the support then it has to be notified explicitly for its consumer.
Schedule:
The schedule is aligned with the Ussuri cycle milestone[2]. I will add the plan in the release schedule also. Phase-1: Dec 09 - Dec 13 R-22 Ussuri-1 milestone ** Project to start dropping the py2 support along with all the py2 CI jobs. Phase-2: Feb 10 - Feb 14 R-13 Ussuri-2 milestone ** This includes Oslo, QA tools (or any other testing tools), common lib (os-brick), Client library. ** This will give enough time to projects to drop the py2 support. Phase-3: Apr 06 - Apr 10 R-5 Ussuri-3 milestone ** Final audit on Phase-1 and Phase-2 plan and make sure everything is done without breaking anything. This is enough time to measure such break or anything extra to do before ussuri final release.
Other discussions points and agreement:
- Projects want to keep python 2 support and need oslo, QA or any other dependent projects/lib support: ** swift. AI: gmann to reach out to swift team about the plan and exact required things from its dependency (the common lib/testing tool).
I chated with timburke on IRC about things required by swift to keep the py2.7 support[1]. Below are client lib/middleware swift required for py2 testing. @timburke, feel free to update if any missing point.
- devstack. able to keep running swift on py2 and rest all services can be on py3
- keystonemiddleware and its dependency
- keystoneclient and openstackclient (dep of keystonemiddleware)
- castellan and barbicanclient
As those lib/middleware going to drop the py2.7 support in phase-2, we need to cap them for swift. I think capping them for python2.7 in upper constraint file would not affect any other users but Matthew Thode can explain better how that will work from the requirement constraint perspective.
[1] http://eavesdrop.openstack.org/irclogs/%23openstack-swift/%23openstack-swift...
-gmann
ya, there are examples already for libs that have dropped py2 support. What you need to do is update global requirements to be something like the following.
sphinx!=1.6.6,!=1.6.7,<2.0.0;python_version=='2.7' # BSD sphinx!=1.6.6,!=1.6.7,!=2.1.0;python_version>='3.4' # BSD
or
keyring<19.0.0;python_version=='2.7' # MIT/PSF keyring;python_version>='3.4' # MIT/PSF
on a related note os-vif is blocked form running tempest jobs under python 3 until https://review.opendev.org/#/c/681029/ is merged due to https://zuul.opendev.org/t/openstack/build/4ff60d6bd2f24782abeb12cc7bdb8013/...
i think this issue will affect any job that install proejcts that use privsep using the required-proejcts section of the zuul job definition. adding a project to required-proejcts sechtion adds it to the LIBS_FROM_GIT varible in devstack. this inturn istalls it twice due to https://review.opendev.org/#/c/418135/ . the side effect of this is that the privsep helper script gets installed under python2 and the neutron ageint in this case gets install under python 3 so when it trys to spawn the privsep deamon and invoke commands it typically expodes due to dependcy issues or in this case because it failed to drop privileges correctly.
so as part of phase 1 we need to merge https://review.opendev.org/#/c/681029/ so that lib project that use required- projects to run with master of project that comsume it and support depends-on can move to python 3 tempest jobs.
Thanks for raising this. I agree on not falling back to py2 in Ussuri, I approved 681029.
-gmann
Hello Everyone,
I would like to notify all projects about important discussions and agreement happened today to move forward for cross projects dependencies and devstack default installation or py2 drop work.
* It is now an official community goal for ussuri[1] and except Swift, all projects agreed to drop py2 as per schedule in goal.
* If any project (openstack services which are planned to drop from now till m-1) drop the py2 with removing the py2 requirements and min python version in setup.cfg which makes that project uninstallable in cross projects jobs then:
Options 1 (suggested): broken projects has to drop the py2 support/testing immediately.
Options 2: if it breaks most of the projects, for example, nova or any other default projects become uninstallable on py2 then we can half-revert[2] the changes from the project caused the break and wait till m1 to merge them back.
* Making Devstack to py3 by default TODAY (otherwise it can break gate everyday). **Devstack default is py2 currently and it was planned to make py3 by default after m-1. But after seeing today gate break, it is hard to maintain devstack-py2-by-default. because projects are dropping the py2 support and devstack py2 by default cause the problem[3]. Today it is from nova side and It can happen due to any projects dropping py2 or I should say it can happen every day as py2 drop patches get merged. ** I am ok to make Devstack py3 by default today which is this patch - https://review.opendev.org/#/c/649097/ ** Action for projects who want to keep testing py2 job till m-1 or whenever they plan to drop py2: Explicitly disable the py3 in their py2 jobs (USE_PYTHON3: False).
* I have pushed the py2 drop patches on almost all the OpenStack services[4] which migrate py2 jobs to py3, remove the py2 requirement but do not mention the min python version in setup.cfg (which can ben done by followup if projects want to do that). I will suggest to merge them asap to avoid any gate break due to cross projects dependency.
[1] https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html [2] half-revert means only change in requirements.txt and setup.cfg - https://review.opendev.org/#/c/695007/ [3] https://bugs.launchpad.net/nova/+bug/1853166 [4] https://review.opendev.org/#/q/topic:drop-py27-support+(status:open+OR+statu...)
-gmann
---- On Wed, 30 Oct 2019 12:03:33 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
---- On Wed, 30 Oct 2019 06:59:19 -0500 Sean Mooney smooney@redhat.com wrote ----
On Tue, 2019-10-29 at 19:40 -0500, Matthew Thode wrote:
On 19-10-29 14:53:11, Ghanshyam Mann wrote:
---- On Thu, 24 Oct 2019 14:32:03 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Hello Everyone,
We had good amount of discussion on the final plan and schedule in today's TC office hour[1].
I captured the agreement on each point in etherpad (you can see the AGREE:). Also summarizing the discussions here. Imp point is if your projects are planning to keep the py2.7 support then do not delay to tell us. Reply on this ML thread or add your project in etherpad.
- Projects can start dropping the py2.7 support. Common lib and testing tools need to wait until milestone-2. ** pepe8 job to be included in openstack-python3-ussuri-jobs-* templates -
https://review.opendev.org/#/c/688997/
** You can drop openstack-python-jobs template and start using ussuri template once 688997 patch is merged. ** Cross projects dependency (if any ) can be sync up among dependent projects.
I will add this plan and schedule as a community goal. The goal is more about what all things to do and when. ** If any project keeping the support then it has to be notified explicitly for its consumer.
Schedule:
The schedule is aligned with the Ussuri cycle milestone[2]. I will add the plan in the release schedule also. Phase-1: Dec 09 - Dec 13 R-22 Ussuri-1 milestone ** Project to start dropping the py2 support along with all the py2 CI jobs. Phase-2: Feb 10 - Feb 14 R-13 Ussuri-2 milestone ** This includes Oslo, QA tools (or any other testing tools), common lib (os-brick), Client library. ** This will give enough time to projects to drop the py2 support. Phase-3: Apr 06 - Apr 10 R-5 Ussuri-3 milestone ** Final audit on Phase-1 and Phase-2 plan and make sure everything is done without breaking anything. This is enough time to measure such break or anything extra to do before ussuri final release.
Other discussions points and agreement:
- Projects want to keep python 2 support and need oslo, QA or any other dependent projects/lib support: ** swift. AI: gmann to reach out to swift team about the plan and exact required things from its dependency (the common lib/testing tool).
I chated with timburke on IRC about things required by swift to keep the py2.7 support[1]. Below are client lib/middleware swift required for py2 testing. @timburke, feel free to update if any missing point.
- devstack. able to keep running swift on py2 and rest all services can be on py3
- keystonemiddleware and its dependency
- keystoneclient and openstackclient (dep of keystonemiddleware)
- castellan and barbicanclient
As those lib/middleware going to drop the py2.7 support in phase-2, we need to cap them for swift. I think capping them for python2.7 in upper constraint file would not affect any other users but Matthew Thode can explain better how that will work from the requirement constraint perspective.
[1] http://eavesdrop.openstack.org/irclogs/%23openstack-swift/%23openstack-swift...
-gmann
ya, there are examples already for libs that have dropped py2 support. What you need to do is update global requirements to be something like the following.
sphinx!=1.6.6,!=1.6.7,<2.0.0;python_version=='2.7' # BSD sphinx!=1.6.6,!=1.6.7,!=2.1.0;python_version>='3.4' # BSD
or
keyring<19.0.0;python_version=='2.7' # MIT/PSF keyring;python_version>='3.4' # MIT/PSF
on a related note os-vif is blocked form running tempest jobs under python 3 until https://review.opendev.org/#/c/681029/ is merged due to https://zuul.opendev.org/t/openstack/build/4ff60d6bd2f24782abeb12cc7bdb8013/...
i think this issue will affect any job that install proejcts that use privsep using the required-proejcts section of the zuul job definition. adding a project to required-proejcts sechtion adds it to the LIBS_FROM_GIT varible in devstack. this inturn istalls it twice due to https://review.opendev.org/#/c/418135/ . the side effect of this is that the privsep helper script gets installed under python2 and the neutron ageint in this case gets install under python 3 so when it trys to spawn the privsep deamon and invoke commands it typically expodes due to dependcy issues or in this case because it failed to drop privileges correctly.
so as part of phase 1 we need to merge https://review.opendev.org/#/c/681029/ so that lib project that use required- projects to run with master of project that comsume it and support depends-on can move to python 3 tempest jobs.
Thanks for raising this. I agree on not falling back to py2 in Ussuri, I approved 681029.
-gmann
---- On Tue, 19 Nov 2019 10:31:51 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Hello Everyone,
I would like to notify all projects about important discussions and agreement happened today to move forward for cross projects dependencies and devstack default installation or py2 drop work.
It is now an official community goal for ussuri[1] and except Swift, all projects agreed to drop py2 as per schedule in goal.
If any project (openstack services which are planned to drop from now till m-1) drop the py2 with removing the py2 requirements and min python version in setup.cfg which makes that project uninstallable in cross projects jobs then:
Options 1 (suggested): broken projects has to drop the py2 support/testing immediately.
Options 2: if it breaks most of the projects, for example, nova or any other default projects become uninstallable on py2 then we can half-revert[2] the changes from the project caused the break and wait till m1 to merge them back.
Making Devstack to py3 by default TODAY (otherwise it can break gate everyday).
**Devstack default is py2 currently and it was planned to make py3 by default after m-1. But after seeing today gate break, it is hard to maintain devstack-py2-by-default. because projects are dropping the py2 support and devstack py2 by default cause the problem[3]. Today it is from nova side and It can happen due to any projects dropping py2 or I should say it can happen every day as py2 drop patches get merged. ** I am ok to make Devstack py3 by default today which is this patch - https://review.opendev.org/#/c/649097/ ** Action for projects who want to keep testing py2 job till m-1 or whenever they plan to drop py2: Explicitly disable the py3 in their py2 jobs (USE_PYTHON3: False).
Devstack patch moving to py3 by default is approved and waiting for base patches to merge first. I am monitoring that to be merged by night.
One thing it will break for sure is grenade jobs which we have seen in the same patch also. After auditing all the projects for grenade py2 jobs, patches to move those jobs to py3 is up[1]. grenade jobs in openstack-zuul-jobs are mainly for stable branches so I kept them on py2 by explicitly disable the py3. later while dropping the py2, I will move those jobs to projects side with py3 version.
If you find your project gate broken for grenade job then, you need to merge the patches which are already up[1].
If any other job is broken then,
Option1: migrate the failing py2 jobs to py3 with 'USE_PYTHON3: True in zuulv3 jobs' and 'DEVSTACK_GATE_USE_PYTHON3=True in legacy jobs'.
Options2: If the migration to py3 takes time due to any reason then restore those jobs to py2 by explicitly disabled the py3 via above variables.
[1] https://review.opendev.org/#/q/topic:drop-py27-support-devstack-default-py3+...)
-gmann
- I have pushed the py2 drop patches on almost all the OpenStack services[4] which migrate py2 jobs to py3, remove the py2 requirement but do not mention t
he min python version in setup.cfg (which can ben done by followup if projects want to
do that). I will suggest to merge them asap to avoid any gate break due to cross projects dependency.
[1] https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html [2] half-revert means only change in requirements.txt and setup.cfg - https://review.opendev.org/#/c/695007/ [3] https://bugs.launchpad.net/nova/+bug/1853166 [4] https://review.opendev.org/#/q/topic:drop-py27-support+(status:open+OR+statu...)
-gmann
---- On Wed, 30 Oct 2019 12:03:33 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
---- On Wed, 30 Oct 2019 06:59:19 -0500 Sean Mooney smooney@redhat.com wrote ----
On Tue, 2019-10-29 at 19:40 -0500, Matthew Thode wrote:
On 19-10-29 14:53:11, Ghanshyam Mann wrote:
---- On Thu, 24 Oct 2019 14:32:03 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Hello Everyone,
We had good amount of discussion on the final plan and schedule in today's TC office hour[1].
I captured the agreement on each point in etherpad (you can see the AGREE:). Also summarizing the discussions here. Imp point is if your projects are planning to keep the py2.7 support then do not delay to tell us. Reply on this ML thread or add your project in etherpad.
- Projects can start dropping the py2.7 support. Common lib and testing tools need to wait until milestone-2. ** pepe8 job to be included in openstack-python3-ussuri-jobs-* templates -
https://review.opendev.org/#/c/688997/
** You can drop openstack-python-jobs template and start using ussuri template once 688997 patch is merged. ** Cross projects dependency (if any ) can be sync up among dependent projects.
I will add this plan and schedule as a community goal. The goal is more about what all things to do and when. ** If any project keeping the support then it has to be notified explicitly for its consumer.
Schedule:
The schedule is aligned with the Ussuri cycle milestone[2]. I will add the plan in the release schedule also. Phase-1: Dec 09 - Dec 13 R-22 Ussuri-1 milestone ** Project to start dropping the py2 support along with all the py2 CI jobs. Phase-2: Feb 10 - Feb 14 R-13 Ussuri-2 milestone ** This includes Oslo, QA tools (or any other testing tools), common lib (os-brick), Client library. ** This will give enough time to projects to drop the py2 support. Phase-3: Apr 06 - Apr 10 R-5 Ussuri-3 milestone ** Final audit on Phase-1 and Phase-2 plan and make sure everything is done without breaking anything. This is enough time to measure such break or anything extra to do before ussuri final release.
Other discussions points and agreement:
- Projects want to keep python 2 support and need oslo, QA or any other dependent projects/lib support: ** swift. AI: gmann to reach out to swift team about the plan and exact required things from its dependency (the common lib/testing tool).
I chated with timburke on IRC about things required by swift to keep the py2.7 support[1]. Below are client lib/middleware swift required for py2 testing. @timburke, feel free to update if any missing point.
- devstack. able to keep running swift on py2 and rest all services can be on py3
- keystonemiddleware and its dependency
- keystoneclient and openstackclient (dep of keystonemiddleware)
- castellan and barbicanclient
As those lib/middleware going to drop the py2.7 support in phase-2, we need to cap them for swift. I think capping them for python2.7 in upper constraint file would not affect any other users but Matthew Thode can explain better how that will work from the requirement constraint perspective.
[1] http://eavesdrop.openstack.org/irclogs/%23openstack-swift/%23openstack-swift...
-gmann
ya, there are examples already for libs that have dropped py2 support. What you need to do is update global requirements to be something like the following.
sphinx!=1.6.6,!=1.6.7,<2.0.0;python_version=='2.7' # BSD sphinx!=1.6.6,!=1.6.7,!=2.1.0;python_version>='3.4' # BSD
or
keyring<19.0.0;python_version=='2.7' # MIT/PSF keyring;python_version>='3.4' # MIT/PSF
on a related note os-vif is blocked form running tempest jobs under python 3 until https://review.opendev.org/#/c/681029/ is merged due to https://zuul.opendev.org/t/openstack/build/4ff60d6bd2f24782abeb12cc7bdb8013/...
i think this issue will affect any job that install proejcts that use privsep using the required-proejcts section of the zuul job definition. adding a project to required-proejcts sechtion adds it to the LIBS_FROM_GIT varible in devstack. this inturn istalls it twice due to https://review.opendev.org/#/c/418135/ . the side effect of this is that the privsep helper script gets installed under python2 and the neutron ageint in this case gets install under python 3 so when it trys to spawn the privsep deamon and invoke commands it typically expodes due to dependcy issues or in this case because it failed to drop privileges correctly.
so as part of phase 1 we need to merge https://review.opendev.org/#/c/681029/ so that lib project that use required- projects to run with master of project that comsume it and support depends-on can move to python 3 tempest jobs.
Thanks for raising this. I agree on not falling back to py2 in Ussuri, I approved 681029.
-gmann
---- On Tue, 19 Nov 2019 19:30:52 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
---- On Tue, 19 Nov 2019 10:31:51 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Hello Everyone,
I would like to notify all projects about important discussions and agreement happened today to move forward for cross projects dependencies and devstack default installation or py2 drop work.
It is now an official community goal for ussuri[1] and except Swift, all projects agreed to drop py2 as per schedule in goal.
If any project (openstack services which are planned to drop from now till m-1) drop the py2 with removing the py2 requirements and min python version in setup.cfg which makes that project uninstallable in cross projects jobs then:
Options 1 (suggested): broken projects has to drop the py2 support/testing immediately.
Options 2: if it breaks most of the projects, for example, nova or any other default projects become uninstallable on py2 then we can half-revert[2] the changes from the project caused the break and wait till m1 to merge them back.
Making Devstack to py3 by default TODAY (otherwise it can break gate everyday).
**Devstack default is py2 currently and it was planned to make py3 by default after m-1. But after seeing today gate break, it is hard to maintain devstack-py2-by-default. because projects are dropping the py2 support and devstack py2 by default cause the problem[3]. Today it is from nova side and It can happen due to any projects dropping py2 or I should say it can happen every day as py2 drop patches get merged. ** I am ok to make Devstack py3 by default today which is this patch - https://review.opendev.org/#/c/649097/ ** Action for projects who want to keep testing py2 job till m-1 or whenever they plan to drop py2: Explicitly disable the py3 in their py2 jobs (USE_PYTHON3: False).
Devstack patch moving to py3 by default is approved and waiting for base patches to merge first. I am monitoring that to be merged by night.
One thing it will break for sure is grenade jobs which we have seen in the same patch also. After auditing all the projects for grenade py2 jobs, patches to move those jobs to py3 is up[1]. grenade jobs in openstack-zuul-jobs are mainly for stable branches so I kept them on py2 by explicitly disable the py3. later while dropping the py2, I will move those jobs to projects side with py3 version.
If you find your project gate broken for grenade job then, you need to merge the patches which are already up[1].
If any other job is broken then,
Option1: migrate the failing py2 jobs to py3 with 'USE_PYTHON3: True in zuulv3 jobs' and 'DEVSTACK_GATE_USE_PYTHON3=True in legacy jobs'.
Options2: If the migration to py3 takes time due to any reason then restore those jobs to py2 by explicitly disabled the py3 via above variables.
[1] https://review.opendev.org/#/q/topic:drop-py27-support-devstack-default-py3+...)
Devstack patch is about to merge, Below is the status of grenade jobs fixes:
* heat- https://review.opendev.org/#/c/695088/ - One test is failing on grenade job consistently. Not sure how that is related to py3.
* barbican- https://review.opendev.org/#/c/695052/ - Some strange behaviour is happening in this, barbican grenade job run stable/train run.yaml always which is stoping this job to move to py3
* Octavia- https://review.opendev.org/#/c/693486/3 - This needs to be updated to mention the py3 version in run.yaml.
* rest all other patches are passing gate so merge them asap before devstack patch is merged and block their gate.
-gmann
-gmann
- I have pushed the py2 drop patches on almost all the OpenStack services[4] which migrate py2 jobs to py3, remove the py2 requirement but do not mention t
he min python version in setup.cfg (which can ben done by followup if projects want to
do that). I will suggest to merge them asap to avoid any gate break due to cross projects dependency.
[1] https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html [2] half-revert means only change in requirements.txt and setup.cfg - https://review.opendev.org/#/c/695007/ [3] https://bugs.launchpad.net/nova/+bug/1853166 [4] https://review.opendev.org/#/q/topic:drop-py27-support+(status:open+OR+statu...)
-gmann
---- On Wed, 30 Oct 2019 12:03:33 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
---- On Wed, 30 Oct 2019 06:59:19 -0500 Sean Mooney smooney@redhat.com wrote ----
On Tue, 2019-10-29 at 19:40 -0500, Matthew Thode wrote:
On 19-10-29 14:53:11, Ghanshyam Mann wrote:
---- On Thu, 24 Oct 2019 14:32:03 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ---- > Hello Everyone, > > We had good amount of discussion on the final plan and schedule in today's TC office hour[1]. > > I captured the agreement on each point in etherpad (you can see the AGREE:). Also summarizing > the discussions here. Imp point is if your projects are planning to keep the py2.7 support then do not delay > to tell us. Reply on this ML thread or add your project in etherpad. > > - Projects can start dropping the py2.7 support. Common lib and testing tools need to wait until milestone-2. > ** pepe8 job to be included in openstack-python3-ussuri-jobs-* templates - https://review.opendev.org/#/c/688997/ > ** You can drop openstack-python-jobs template and start using ussuri template once 688997 patch is merged. > ** Cross projects dependency (if any ) can be sync up among dependent projects. > > - I will add this plan and schedule as a community goal. The goal is more about what all things to do and when. > ** If any project keeping the support then it has to be notified explicitly for its consumer. > > - Schedule: > The schedule is aligned with the Ussuri cycle milestone[2]. I will add the plan in the release schedule also. > Phase-1: Dec 09 - Dec 13 R-22 Ussuri-1 milestone > ** Project to start dropping the py2 support along with all the py2 CI jobs. > Phase-2: Feb 10 - Feb 14 R-13 Ussuri-2 milestone > ** This includes Oslo, QA tools (or any other testing tools), common lib (os-brick), Client library. > ** This will give enough time to projects to drop the py2 support. > Phase-3: Apr 06 - Apr 10 R-5 Ussuri-3 milestone > ** Final audit on Phase-1 and Phase-2 plan and make sure everything is done without breaking anything. > This is enough time to measure such break or anything extra to do before ussuri final release. > > Other discussions points and agreement: > - Projects want to keep python 2 support and need oslo, QA or any other dependent projects/lib support: > ** swift. AI: gmann to reach out to swift team about the plan and exact required things from its dependency > (the common lib/testing tool).
I chated with timburke on IRC about things required by swift to keep the py2.7 support[1]. Below are client lib/middleware swift required for py2 testing. @timburke, feel free to update if any missing point.
- devstack. able to keep running swift on py2 and rest all services can be on py3
- keystonemiddleware and its dependency
- keystoneclient and openstackclient (dep of keystonemiddleware)
- castellan and barbicanclient
As those lib/middleware going to drop the py2.7 support in phase-2, we need to cap them for swift. I think capping them for python2.7 in upper constraint file would not affect any other users but Matthew Thode can explain better how that will work from the requirement constraint perspective.
[1] http://eavesdrop.openstack.org/irclogs/%23openstack-swift/%23openstack-swift...
-gmann
ya, there are examples already for libs that have dropped py2 support. What you need to do is update global requirements to be something like the following.
sphinx!=1.6.6,!=1.6.7,<2.0.0;python_version=='2.7' # BSD sphinx!=1.6.6,!=1.6.7,!=2.1.0;python_version>='3.4' # BSD
or
keyring<19.0.0;python_version=='2.7' # MIT/PSF keyring;python_version>='3.4' # MIT/PSF
on a related note os-vif is blocked form running tempest jobs under python 3 until https://review.opendev.org/#/c/681029/ is merged due to https://zuul.opendev.org/t/openstack/build/4ff60d6bd2f24782abeb12cc7bdb8013/...
i think this issue will affect any job that install proejcts that use privsep using the required-proejcts section of the zuul job definition. adding a project to required-proejcts sechtion adds it to the LIBS_FROM_GIT varible in devstack. this inturn istalls it twice due to https://review.opendev.org/#/c/418135/ . the side effect of this is that the privsep helper script gets installed under python2 and the neutron ageint in this case gets install under python 3 so when it trys to spawn the privsep deamon and invoke commands it typically expodes due to dependcy issues or in this case because it failed to drop privileges correctly.
so as part of phase 1 we need to merge https://review.opendev.org/#/c/681029/ so that lib project that use required- projects to run with master of project that comsume it and support depends-on can move to python 3 tempest jobs.
Thanks for raising this. I agree on not falling back to py2 in Ussuri, I approved 681029.
-gmann
On 20/11/2019 09.08, Ghanshyam Mann wrote:
[...]
- barbican- https://review.opendev.org/#/c/695052/
- Some strange behaviour is happening in this, barbican grenade job run stable/train run.yaml always which is stoping this job to move to py3
AFAIU, the stable jobs have a branch matcher, see https://opendev.org/openstack/barbican/src/branch/stable/train/.zuul.yaml#L1... - and thus apply.
You need https://review.opendev.org/689458 and friends merged for all branches to fix this,
Andreas
---- On Wed, 20 Nov 2019 04:30:06 -0600 Andreas Jaeger aj@suse.com wrote ----
On 20/11/2019 09.08, Ghanshyam Mann wrote:
[...]
- barbican- https://review.opendev.org/#/c/695052/
- Some strange behaviour is happening in this, barbican grenade job run stable/train run.yaml always which is stoping this job to move to py3
AFAIU, the stable jobs have a branch matcher, see https://opendev.org/openstack/barbican/src/branch/stable/train/.zuul.yaml#L1...
- and thus apply.
You need https://review.opendev.org/689458 and friends merged for all branches to fix this,
Thanks Andreas, that was something i was suspecting but could not understand how zuul picked the job inventory from stable/train every time and not from master where the master is branchless so should be eligible for the master gate.
One more thing, only run.yaml is taken from stable/train and rest all like pre.yaml etc is from the master branch. This is another the mystery I would like to learn :).
-gmann
Andreas
Andreas Jaeger aj@suse.com Twitter: jaegerandi SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D 90409 Nürnberg (HRB 36809, AG Nürnberg) GF: Felix Imendörffer GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
---- On Wed, 20 Nov 2019 10:15:10 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
---- On Wed, 20 Nov 2019 04:30:06 -0600 Andreas Jaeger aj@suse.com wrote ----
On 20/11/2019 09.08, Ghanshyam Mann wrote:
[...]
- barbican- https://review.opendev.org/#/c/695052/
- Some strange behaviour is happening in this, barbican grenade job run stable/train run.yaml always which is stoping this job to move to py3
Devstck patch is merged now and it defaults to py3.
below are the pending fix for grenade jobs which will fail now. Request the corresponding project to merge them.
- https://review.opendev.org/#/q/topic:drop-py27-support-devstack-default-py3+...
If any other job starts failing then below are the options to fix them:
Option1: migrate the failing py2 jobs to py3 with 'USE_PYTHON3: True in zuulv3 jobs' and 'DEVSTACK_GATE_USE_PYTHON3=True in legacy jobs'.
Options2: If the migration to py3 takes time due to any reason then restore those jobs to py2 by explicitly disabled the py3 via above variables.
-gmann
AFAIU, the stable jobs have a branch matcher, see https://opendev.org/openstack/barbican/src/branch/stable/train/.zuul.yaml#L1...
- and thus apply.
You need https://review.opendev.org/689458 and friends merged for all branches to fix this,
Thanks Andreas, that was something i was suspecting but could not understand how zuul picked the job inventory from stable/train every time and not from master where the master is branchless so should be eligible for the master gate.
One more thing, only run.yaml is taken from stable/train and rest all like pre.yaml etc is from the master branch. This is another the mystery I would like to learn :).
-gmann
Andreas
Andreas Jaeger aj@suse.com Twitter: jaegerandi SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D 90409 Nürnberg (HRB 36809, AG Nürnberg) GF: Felix Imendörffer GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
participants (7)
-
Andreas Jaeger
-
Ghanshyam Mann
-
Goutham Pacha Ravi
-
Jeremy Stanley
-
Matt Riedemann
-
Matthew Thode
-
Sean Mooney