[neutron][tempest][all] Broken CI, any job inherited from "devstack"
Hello all: I broke the OpenStack CI (good start as Neutron PTL). I pushed [1] testing only against the Neutron CI. After making the needed changes [2], I thought any other job would be safe. I've opened [3]. The default OVN version installed in the CI, using Ubuntu 20.04, is v20.03, that is a bit old. I've proposed to bump to v21.06, extensively tested in the Neutron CI. Any tempest job from any project is inherited from this one. Once we have migrated to Ubuntu 22.04 during this cycle, we'll remove this forced OVN installation from source. Regards. [1]https://review.opendev.org/c/openstack/neutron/+/859642 [2]https://review.opendev.org/c/openstack/neutron/+/860078/ [3]https://bugs.launchpad.net/devstack/+bug/1991952 [4]https://review.opendev.org/c/openstack/devstack/+/860577
Hello: I've filled [1]. We'll revert the Neutron patch. Regards. [1]https://bugs.launchpad.net/neutron/+bug/1991962 On Thu, Oct 6, 2022 at 5:08 PM Rodolfo Alonso Hernandez <ralonsoh@redhat.com> wrote:
Hello all:
I broke the OpenStack CI (good start as Neutron PTL). I pushed [1] testing only against the Neutron CI. After making the needed changes [2], I thought any other job would be safe.
I've opened [3]. The default OVN version installed in the CI, using Ubuntu 20.04, is v20.03, that is a bit old. I've proposed to bump to v21.06, extensively tested in the Neutron CI. Any tempest job from any project is inherited from this one.
Once we have migrated to Ubuntu 22.04 during this cycle, we'll remove this forced OVN installation from source.
Regards.
[1]https://review.opendev.org/c/openstack/neutron/+/859642 [2]https://review.opendev.org/c/openstack/neutron/+/860078/ [3]https://bugs.launchpad.net/devstack/+bug/1991952 [4]https://review.opendev.org/c/openstack/devstack/+/860577
On 2022-10-06 17:08:18 +0200 (+0200), Rodolfo Alonso Hernandez wrote: [...]
I've opened [3]. The default OVN version installed in the CI, using Ubuntu 20.04, is v20.03, that is a bit old. I've proposed to bump to v21.06, extensively tested in the Neutron CI. Any tempest job from any project is inherited from this one.
Once we have migrated to Ubuntu 22.04 during this cycle, we'll remove this forced OVN installation from source. [...]
What's the implication for upgrades? Historically, we've needed the software to be operable on the prior platform (upgrade OpenStack from Zed to 2023.1/Antelope, then upgrade Ubuntu from Focal to Jammy). Now with SLURP in the picture, we'll even need OpenStack 2023.2/B working on Focal before upgrading to Jammy, right? -- Jeremy Stanley
I don't see in the documentation (probably I didn't check all of it) that an OpenStack version upgrade doesn't imply other library/modules/services upgrade. In this case, you will need to bump the OVN/OVS version, keeping the same OVS/NB/SB information (probably not the same database structures). In any case, I'll try to make the Neutron server compatible with both scenarios, but that will take some time to implement (if possible). On Thu, Oct 6, 2022 at 6:13 PM Jeremy Stanley <fungi@yuggoth.org> wrote:
I've opened [3]. The default OVN version installed in the CI, using Ubuntu 20.04, is v20.03, that is a bit old. I've proposed to bump to v21.06, extensively tested in the Neutron CI. Any tempest job from any project is inherited from this one.
Once we have migrated to Ubuntu 22.04 during this cycle, we'll remove
On 2022-10-06 17:08:18 +0200 (+0200), Rodolfo Alonso Hernandez wrote: [...] this
forced OVN installation from source. [...]
What's the implication for upgrades? Historically, we've needed the software to be operable on the prior platform (upgrade OpenStack from Zed to 2023.1/Antelope, then upgrade Ubuntu from Focal to Jammy). Now with SLURP in the picture, we'll even need OpenStack 2023.2/B working on Focal before upgrading to Jammy, right? -- Jeremy Stanley
On Thu, Oct 6, 2022, at 9:24 AM, Rodolfo Alonso Hernandez wrote:
I don't see in the documentation (probably I didn't check all of it) that an OpenStack version upgrade doesn't imply other library/modules/services upgrade. In this case, you will need to bump the OVN/OVS version, keeping the same OVS/NB/SB information (probably not the same database structures).
I don't know about documentation (but that may have come along with the new SLURP stuff), but Grenade enforces this sort of thing in CI. When we run Grenade against a branch we install the old version of OpenStack on the platform that the old version of OpenStack was tested on and then upgrade to the new current version of OpenStack. This upgrade is done on a host without upgrading the host itself.
In any case, I'll try to make the Neutron server compatible with both scenarios, but that will take some time to implement (if possible).
On Thu, Oct 6, 2022 at 6:13 PM Jeremy Stanley <fungi@yuggoth.org> wrote:
On 2022-10-06 17:08:18 +0200 (+0200), Rodolfo Alonso Hernandez wrote: [...]
I've opened [3]. The default OVN version installed in the CI, using Ubuntu 20.04, is v20.03, that is a bit old. I've proposed to bump to v21.06, extensively tested in the Neutron CI. Any tempest job from any project is inherited from this one.
Once we have migrated to Ubuntu 22.04 during this cycle, we'll remove this forced OVN installation from source. [...]
What's the implication for upgrades? Historically, we've needed the software to be operable on the prior platform (upgrade OpenStack from Zed to 2023.1/Antelope, then upgrade Ubuntu from Focal to Jammy). Now with SLURP in the picture, we'll even need OpenStack 2023.2/B working on Focal before upgrading to Jammy, right? -- Jeremy Stanley
I don't know about documentation (but that may have come along with the new SLURP stuff), but Grenade enforces this sort of thing in CI. When we run Grenade against a branch we install the old version of OpenStack on the platform that the old version of OpenStack was tested on and then upgrade to the new current version of OpenStack. This upgrade is done on a host without upgrading the host itself.
Yeah, I think this is "expectation by code" in that grenade requires that to work, so (assuming you're running those jobs) you'll be forced into that support. Perhaps we need to expand on that a bit in words to make sure it's clear. --Dan
On 2022-10-06 09:55:30 -0700 (-0700), Dan Smith wrote: [...]
Yeah, I think this is "expectation by code" in that grenade requires that to work, so (assuming you're running those jobs) you'll be forced into that support. Perhaps we need to expand on that a bit in words to make sure it's clear.
One fairly lightweight solution would just be to include this in the cycle-specific PTI doc. So for the 2023.1 tested runtime list both Ubuntu 20.04 and Ubuntu 22.04 (maybe with a boilerplate sentence that the former platform is only tested insofar as to support in-place upgrading of OpenStack software before upgrading to the latter platform in that release). -- Jeremy Stanley
---- On Thu, 06 Oct 2022 10:06:55 -0700 Jeremy Stanley wrote ---
On 2022-10-06 09:55:30 -0700 (-0700), Dan Smith wrote: [...]
Yeah, I think this is "expectation by code" in that grenade requires that to work, so (assuming you're running those jobs) you'll be forced into that support. Perhaps we need to expand on that a bit in words to make sure it's clear.
One fairly lightweight solution would just be to include this in the cycle-specific PTI doc. So for the 2023.1 tested runtime list both Ubuntu 20.04 and Ubuntu 22.04 (maybe with a boilerplate sentence that the former platform is only tested insofar as to support in-place upgrading of OpenStack software before upgrading to the latter platform in that release).
yeah, we do test it but I agree to document it somewhere in PTI, I started the documentation, feel free to review/feedback if more information needs to be mentioned - https://review.opendev.org/c/openstack/governance/+/860599 -gmann
-- Jeremy Stanley
On 2022-10-06 16:12:41 +0000 (+0000), Jeremy Stanley wrote: [...]
Now with SLURP in the picture, we'll even need OpenStack 2023.2/B working on Focal before upgrading to Jammy, right?
Just to correct myself, Zed to 2023.2/B is not SLURP. If this transition were occurring between 2023.1/Antelope and 2023.2/B then we'd also need to solve it for upgrades from 2023.1/Antelope to 2024.1/C, but that's thankfully not the case this time. -- Jeremy Stanley
participants (5)
-
Clark Boylan
-
Dan Smith
-
Ghanshyam Mann
-
Jeremy Stanley
-
Rodolfo Alonso Hernandez