[qa] [all] Opinion on dropping the py2.7 support from Tempest & Tempest plugin
Hello Everyone, We are in Ussuri development cycle which is planned to drop the py2.7 support[1]. I was holding and planning to support the py2.7 in Tempest due to its branchless model[2]. My main point to keep py2 support is if any users running the Tempest on py27 env cloud and not in virtual env then they can keep running in the same way. But this might be just me overthinking on this usage. What happens if we drop py2.7 from Tempest: * Users with the above case have the way to install the latest Tempest on virtual env of py3. or use the Tempest tag if they do not need latest Tempest. * No change in users using Tempest on py3 or in py3 virtual env. * Upstream testing of master or stable branch is no issue as we install the Tempest in virtual env. Tempest in py3 venv can always test the py2.7 jobs. * other than that no change in Tempest usage or at least it would not break anything. Why we cannot keep the support: There is no big cost of supporting the py2.7 in Tempest itself but that require Temepst dependency (OpenStack lib like Oslo and non-openstack maintained lib [3]) to keep supporting the py2.7 which is not feasible. Other solution: One way is to cut the Tempest stable branch and keep the py2.7 support there with eligible backport from Tempest master which is py3 only. But I would say, QA team has no bandwidth to do so. if anyone wants to maintain that then we can discuss this option in more detail. I have given a second thought on this and now ok to drop the py2.7 from Tempest by considering all the above points. Please reply if any disagreement on this or add if anything I missed to consider. NOTE: Tempest includes its plugins also. [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.h... [2] https://review.opendev.org/#/c/681203/ [3] https://opendev.org/openstack/tempest/src/commit/7fdd39c6dbde37bccd419c4037e... -gmann
On 10/23/2019 2:08 PM, Ghanshyam Mann wrote:
What happens if we drop py2.7 from Tempest: * Users with the above case have the way to install the latest Tempest on virtual env of py3. or use the Tempest tag if they do not need latest Tempest.
This seems sufficient to me and testing from a tag is what we're doing upstream in stable/ocata and stable/pike branches anyway - not because of python version stuff but because of extended maintenance and backward incompatible changes since those branches which break testing in ocata and pike with tempest from master.
Other solution: One way is to cut the Tempest stable branch and keep the py2.7 support there with eligible backport from Tempest master which is py3 only. But I would say, QA team has no bandwidth to do so. if anyone wants to maintain that then we can discuss this option in more detail.
I would avoid creating a stable branch for tempest if at all possible since we have valid options to workaround it (above) and I just don't think we want toy with that idea and the precedent it could set for relaxing other rules around how tempest is developed. -- Thanks, Matt
On Wed, Oct 23, 2019 at 1:27 PM Matt Riedemann <mriedemos@gmail.com> wrote:
On 10/23/2019 2:08 PM, Ghanshyam Mann wrote:
What happens if we drop py2.7 from Tempest: * Users with the above case have the way to install the latest Tempest on virtual env of py3. or use the Tempest tag if they do not need latest Tempest.
This seems sufficient to me and testing from a tag is what we're doing upstream in stable/ocata and stable/pike branches anyway - not because of python version stuff but because of extended maintenance and backward incompatible changes since those branches which break testing in ocata and pike with tempest from master.
Other solution: One way is to cut the Tempest stable branch and keep the py2.7 support there with eligible backport from Tempest master which is py3 only. But I would say, QA team has no bandwidth to do so. if anyone wants to maintain that then we can discuss this option in more detail.
I would avoid creating a stable branch for tempest if at all possible since we have valid options to workaround it (above) and I just don't think we want toy with that idea and the precedent it could set for relaxing other rules around how tempest is developed.
My concern is that in tripleo/puppet we currently rely on centos7/python2 as centos8 is still not yet available. So this pretty much means we likely won't be able to run the latest tempest anymore and there goes our validations. We could pin to a version (we've had to do that in the past) but I'd be concerned about things that go untested until we can finally get python3 available. I think it might be beneficial to have a py2-em branch similar to what we do when we create -em branches where folks who still have to have python2 wouldn't be completely blocked. Thanks, -Alex
--
Thanks,
Matt
On 10/23/2019 2:36 PM, Alex Schultz wrote:
My concern is that in tripleo/puppet we currently rely on centos7/python2 as centos8 is still not yet available. So this pretty much means we likely won't be able to run the latest tempest anymore and there goes our validations.
How much stuff do you think is going to land in tempest that is going to be useful validation between the time you could use a tagged version of tempest before py27 support is dropped and when you can roll to centos8? I can't imagine it's much and you're still going to get all of the interop style smoke tests we already have. Also, you can't run tempest from a py3 virtual environment or container? -- Thanks, Matt
On Wed, Oct 23, 2019 at 1:40 PM Matt Riedemann <mriedemos@gmail.com> wrote:
On 10/23/2019 2:36 PM, Alex Schultz wrote:
My concern is that in tripleo/puppet we currently rely on centos7/python2 as centos8 is still not yet available. So this pretty much means we likely won't be able to run the latest tempest anymore and there goes our validations.
How much stuff do you think is going to land in tempest that is going to be useful validation between the time you could use a tagged version of tempest before py27 support is dropped and when you can roll to centos8? I can't imagine it's much and you're still going to get all of the interop style smoke tests we already have.
It's probably not going to cause problems, but it's not like we haven't had issues in previous cycles. The concern is really we don't currently have an ETA when RDO will be fully up on CentOS8. So it could be weeks or months. We're hoping for sooner rather than later but once py2 support is officially dropped, who knows what folks might want to try and start working on.
Also, you can't run tempest from a py3 virtual environment or container?
puppet no, tripleo maybe. It's already a container in tripleo and we do have a rhel8 container but the issue is reallly packaging for both.
--
Thanks,
Matt
---- On Wed, 23 Oct 2019 15:06:04 -0500 Alex Schultz <aschultz@redhat.com> wrote ----
On Wed, Oct 23, 2019 at 1:40 PM Matt Riedemann <mriedemos@gmail.com> wrote: On 10/23/2019 2:36 PM, Alex Schultz wrote:
My concern is that in tripleo/puppet we currently rely on centos7/python2 as centos8 is still not yet available. So this pretty much means we likely won't be able to run the latest tempest anymore and there goes our validations.
How much stuff do you think is going to land in tempest that is going to be useful validation between the time you could use a tagged version of tempest before py27 support is dropped and when you can roll to centos8? I can't imagine it's much and you're still going to get all of the interop style smoke tests we already have.
It's probably not going to cause problems, but it's not like we haven't had issues in previous cycles. The concern is really we don't currently have an ETA when RDO will be fully up on CentOS8. So it could be weeks or months. We're hoping for sooner rather than later but once py2 support is officially dropped, who knows what folks might want to try and start working on. Also, you can't run tempest from a py3 virtual environment or container?
A month or so is all fine. Dropping py2 from Tempest is going to be the last in phase-2 (Ussuri milestone-2 ) (final schedule to be discussed tomorrow ). - https://etherpad.openstack.org/p/drop-python2-support -gmann
puppet no, tripleo maybe. It's already a container in tripleo and we do have a rhel8 container but the issue is reallly packaging for both. --
Thanks,
Matt
On Wed, Oct 23, 2019, at 12:36 PM, Alex Schultz wrote:
On Wed, Oct 23, 2019 at 1:27 PM Matt Riedemann <mriedemos@gmail.com> wrote:
On 10/23/2019 2:08 PM, Ghanshyam Mann wrote:
What happens if we drop py2.7 from Tempest: * Users with the above case have the way to install the latest Tempest on virtual env of py3. or use the Tempest tag if they do not need latest Tempest.
This seems sufficient to me and testing from a tag is what we're doing upstream in stable/ocata and stable/pike branches anyway - not because of python version stuff but because of extended maintenance and backward incompatible changes since those branches which break testing in ocata and pike with tempest from master.
Other solution: One way is to cut the Tempest stable branch and keep the py2.7 support there with eligible backport from Tempest master which is py3 only. But I would say, QA team has no bandwidth to do so. if anyone wants to maintain that then we can discuss this option in more detail.
I would avoid creating a stable branch for tempest if at all possible since we have valid options to workaround it (above) and I just don't think we want toy with that idea and the precedent it could set for relaxing other rules around how tempest is developed.
My concern is that in tripleo/puppet we currently rely on centos7/python2 as centos8 is still not yet available. So this pretty much means we likely won't be able to run the latest tempest anymore and there goes our validations. We could pin to a version (we've had to do that in the past) but I'd be concerned about things that go untested until we can finally get python3 available. I think it might be beneficial to have a py2-em branch similar to what we do when we create -em branches where folks who still have to have python2 wouldn't be completely blocked.
The infra team has centos-8 images available now. Another option is to run tempest in a container to host python3. That should work on CentOS 7.
Thanks, -Alex
On Wed, Oct 23, 2019, at 12:36 PM, Alex Schultz wrote:
On Wed, Oct 23, 2019 at 1:27 PM Matt Riedemann <mriedemos@gmail.com> wrote:
On 10/23/2019 2:08 PM, Ghanshyam Mann wrote:
What happens if we drop py2.7 from Tempest: * Users with the above case have the way to install the latest Tempest on virtual env of py3. or use the Tempest tag if they do not need latest Tempest.
This seems sufficient to me and testing from a tag is what we're doing upstream in stable/ocata and stable/pike branches anyway - not because of python version stuff but because of extended maintenance and backward incompatible changes since those branches which break testing in ocata and pike with tempest from master.
Other solution: One way is to cut the Tempest stable branch and keep the py2.7 support there with eligible backport from Tempest master which is py3 only. But I would say, QA team has no bandwidth to do so. if anyone wants to maintain that then we can discuss this option in more detail.
I would avoid creating a stable branch for tempest if at all possible since we have valid options to workaround it (above) and I just don't think we want toy with that idea and the precedent it could set for relaxing other rules around how tempest is developed.
My concern is that in tripleo/puppet we currently rely on centos7/python2 as centos8 is still not yet available. So this pretty much means we likely won't be able to run the latest tempest anymore and there goes our validations. We could pin to a version (we've had to do that in the past) but I'd be concerned about things that go untested until we can finally get python3 available. I think it might be beneficial to have a py2-em branch similar to what we do when we create -em branches where folks who still have to have python2 wouldn't be completely blocked.
The infra team has centos-8 images available now. Another option is to run tempest in a container to host python3. That should work on CentOS 7. you also can install py36 on centos 7 form https://ius.io/
On Wed, 2019-10-23 at 13:07 -0700, Clark Boylan wrote: the "inline with upstream stable" repos https://github.com/iusrepo/python36 and its also available in eple so you can do python 3 testing fine on centos 7 if you need too. by the way when i downloaded the centos-8 image form infra glean did not automaticaly pick up my ssh keys form the openstack metadata serivce or config drive i assuem that is fixed/works in the gate?
Thanks, -Alex
On Wed, Oct 23, 2019, at 1:56 PM, Sean Mooney wrote:
by the way when i downloaded the centos-8 image form infra glean did not automaticaly pick up my ssh keys form the openstack metadata serivce or config drive i assuem that is fixed/works in the gate?
Glean does not support metadata service it only works with the config drive. It must be working in the gate because the images boot and I am able to ssh in. Note that glean configures the ssh key on the 'root' user and not 'centos' or 'ubuntu' or 'fedora'. Clark
On Wed, Oct 23, 2019, at 1:56 PM, Sean Mooney wrote:
by the way when i downloaded the centos-8 image form infra glean did not automaticaly pick up my ssh keys form the openstack metadata serivce or config drive i assuem that is fixed/works in the gate?
Glean does not support metadata service it only works with the config drive. It must be working in the gate because the images boot and I am able to ssh in. Note that glean configures the ssh key on the 'root' user and not 'centos' or 'ubuntu' or 'fedora'. yep that was the issue. if i enable config drive and log in as root it works.
On Wed, 2019-10-23 at 14:06 -0700, Clark Boylan wrote: the only other issue i have found is the centos-8 vms are not getting ipv6 addresss but my ubunut ones are but that is something to look into after. thanks for the tip on root login.
Clark
On 2019-10-23 13:07:21 -0700 (-0700), Clark Boylan wrote:
On Wed, Oct 23, 2019, at 12:36 PM, Alex Schultz wrote: [...]
My concern is that in tripleo/puppet we currently rely on centos7/python2 as centos8 is still not yet available. [...] The infra team has centos-8 images available now. [...]
That was also the first thing which jumped to mind for me, but in another E-mail in the thread Alex clarifies that what's not yet available isn't CentOS 8, but rather a working build of RDO for CentOS 8 (and TripleO is dependent on RDO's packages). -- Jeremy Stanley
On Wed, Oct 23, 2019 at 02:08:38PM -0500, Ghanshyam Mann wrote:
Hello Everyone,
We are in Ussuri development cycle which is planned to drop the py2.7 support[1].
I was holding and planning to support the py2.7 in Tempest due to its branchless model[2]. My main point to keep py2 support is if any users running the Tempest on py27 env cloud and not in virtual env then they can keep running in the same way. But this might be just me overthinking on this usage.
What happens if we drop py2.7 from Tempest: * Users with the above case have the way to install the latest Tempest on virtual env of py3. or use the Tempest tag if they do not need latest Tempest. * No change in users using Tempest on py3 or in py3 virtual env. * Upstream testing of master or stable branch is no issue as we install the Tempest in virtual env. Tempest in py3 venv can always test the py2.7 jobs. * other than that no change in Tempest usage or at least it would not break anything.
I have made this exact argument before about venvs and tempest being not actually part of a cloud installation (just in other contexts). In my experience most people don't actually agree with it for whatever reason (I assume because the installer/deployment projects treat tempest as the same thing as other openstack projects). But I still feel that way and even if some people don't agree there are still releases that support python 2.7. In general I'm in favor of just doing this though, as long as we didn't already say that we'd continue supporting 2.7 somewhere in tempest until the Train EM date. If we did advertise that anywhere then we'll have to provide a deprecation period for those people who could have latched onto that.
Why we cannot keep the support: There is no big cost of supporting the py2.7 in Tempest itself but that require Temepst dependency (OpenStack lib like Oslo and non-openstack maintained lib [3]) to keep supporting the py2.7 which is not feasible.
In my experience the list of requirements for tempest is not crazy long and most of them don't have big api divergance (or at least how tempest uses it). I'd almost say just set a version cap with python_version==2.7 to a requirement when/if a requirement drops support for 2.7. Of course that probably has g-r implications, not sure how that would work.
Other solution: One way is to cut the Tempest stable branch and keep the py2.7 support there with eligible backport from Tempest master which is py3 only. But I would say, QA team has no bandwidth to do so. if anyone wants to maintain that then we can discuss this option in more detail.
Branching doesn't actually fix any of the harms you have outlined above. It just increases the complexity of maintainence.
I have given a second thought on this and now ok to drop the py2.7 from Tempest by considering all the above points. Please reply if any disagreement on this or add if anything I missed to consider.
NOTE: Tempest includes its plugins also.
I don't actually buy this point, a plugin is independently maintained. If the plugin maintainers do not want to support python 3 they don't have to. Just like any other project that has an upstream dep that supports python 2.7 even if they don't support it. -Matt Treinish
[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.h... [2] https://review.opendev.org/#/c/681203/ [3] https://opendev.org/openstack/tempest/src/commit/7fdd39c6dbde37bccd419c4037e...
---- On Wed, 23 Oct 2019 14:34:30 -0500 Matthew Treinish <mtreinish@kortar.org> wrote ----
On Wed, Oct 23, 2019 at 02:08:38PM -0500, Ghanshyam Mann wrote:
Hello Everyone,
We are in Ussuri development cycle which is planned to drop the py2.7 support[1].
I was holding and planning to support the py2.7 in Tempest due to its branchless model[2]. My main point to keep py2 support is if any users running the Tempest on py27 env cloud and not in virtual env then they can keep running in the same way. But this might be just me overthinking on this usage.
What happens if we drop py2.7 from Tempest: * Users with the above case have the way to install the latest Tempest on virtual env of py3. or use the Tempest tag if they do not need latest Tempest. * No change in users using Tempest on py3 or in py3 virtual env. * Upstream testing of master or stable branch is no issue as we install the Tempest in virtual env. Tempest in py3 venv can always test the py2.7 jobs. * other than that no change in Tempest usage or at least it would not break anything.
I have made this exact argument before about venvs and tempest being not actually part of a cloud installation (just in other contexts). In my experience most people don't actually agree with it for whatever reason (I assume because the installer/deployment projects treat tempest as the same thing as other openstack projects). But I still feel that way and even if some people don't agree there are still releases that support python 2.7.
In general I'm in favor of just doing this though, as long as we didn't already say that we'd continue supporting 2.7 somewhere in tempest until the Train EM date. If we did advertise that anywhere then we'll have to provide a deprecation period for those people who could have latched onto that.
I do think we have said or documented anywhere about 2.7 support till Train EM. But sending the warning of py2.7 drop now would be a good idea to notify the people. Because we will droping the support during milestone-2 which is during mid of Feb.
Why we cannot keep the support: There is no big cost of supporting the py2.7 in Tempest itself but that require Temepst dependency (OpenStack lib like Oslo and non-openstack maintained lib [3]) to keep supporting the py2.7 which is not feasible.
In my experience the list of requirements for tempest is not crazy long and most of them don't have big api divergance (or at least how tempest uses it). I'd almost say just set a version cap with python_version==2.7 to a requirement when/if a requirement drops support for 2.7. Of course that probably has g-r implications, not sure how that would work.
Yeah, we could do that if someone strongly says or convience us not to drop the py2.7 from Tempest.
Other solution: One way is to cut the Tempest stable branch and keep the py2.7 support there with eligible backport from Tempest master which is py3 only. But I would say, QA team has no bandwidth to do so. if anyone wants to maintain that then we can discuss this option in more detail.
Branching doesn't actually fix any of the harms you have outlined above. It just increases the complexity of maintainence.
I have given a second thought on this and now ok to drop the py2.7 from Tempest by considering all the above points. Please reply if any disagreement on this or add if anything I missed to consider.
NOTE: Tempest includes its plugins also.
I don't actually buy this point, a plugin is independently maintained. If the plugin maintainers do not want to support python 3 they don't have to. Just like any other project that has an upstream dep that supports python 2.7 even if they don't support it.
Agree. I added plugin also in this scope because most of the plugins wait for guidelines from the Tempest side and they use lot of interfaces from Tempest. They can drop py2.7 independent of Tempest but keeping py2.7 support is not possible if Tempest drop. -gmann
-Matt Treinish
[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.h... [2] https://review.opendev.org/#/c/681203/ [3] https://opendev.org/openstack/tempest/src/commit/7fdd39c6dbde37bccd419c4037e...
participants (7)
-
Alex Schultz
-
Clark Boylan
-
Ghanshyam Mann
-
Jeremy Stanley
-
Matt Riedemann
-
Matthew Treinish
-
Sean Mooney