[TC][nova] Failing device detachments on Focal
Hi,
I want to raise a potential conflict between the fact that OpenStack Victoria officially supports Ubuntu Focal[3] and a high severity regression in Nova [1] caused by a qemu bug [2] in the qemu version that is shipped with Focal.
The Nova team documented[4] it as a limitation without a known workaround in Nova. Also as far as I know Canonical does not support downgrading the qemu version so I don't see a viable workaround on Focal side either.
Is the Nova reno[4] enough from the TC perspective to go forward with the switch to Focal and with the Victoria release?
cheers, gibi
[1] https://bugs.launchpad.net/nova/+bug/1882521 [2] https://bugs.launchpad.net/qemu/+bug/1894804 [3] https://governance.openstack.org/tc/reference/runtimes/victoria.html [4] https://review.opendev.org/#/c/752654
Is there a version of qemu that works on Focal that deployment projects could use now?
-yoctozepto
On Fri, Sep 18, 2020 at 12:16 PM Balázs Gibizer balazs.gibizer@est.tech wrote:
Hi,
I want to raise a potential conflict between the fact that OpenStack Victoria officially supports Ubuntu Focal[3] and a high severity regression in Nova [1] caused by a qemu bug [2] in the qemu version that is shipped with Focal.
The Nova team documented[4] it as a limitation without a known workaround in Nova. Also as far as I know Canonical does not support downgrading the qemu version so I don't see a viable workaround on Focal side either.
Is the Nova reno[4] enough from the TC perspective to go forward with the switch to Focal and with the Victoria release?
cheers, gibi
[1] https://bugs.launchpad.net/nova/+bug/1882521 [2] https://bugs.launchpad.net/qemu/+bug/1894804 [3] https://governance.openstack.org/tc/reference/runtimes/victoria.html [4] https://review.opendev.org/#/c/752654
On Fri, Sep 18, 2020 at 12:20, Radosław Piliszek radoslaw.piliszek@gmail.com wrote:
Is there a version of qemu that works on Focal that deployment projects could use now?
We use the train ubuntu cloud archive to get an unaffected qemu version, which is now 4.0
cheers, gibi
-yoctozepto
On Fri, Sep 18, 2020 at 12:16 PM Balázs Gibizer balazs.gibizer@est.tech wrote:
Hi,
I want to raise a potential conflict between the fact that OpenStack Victoria officially supports Ubuntu Focal[3] and a high severity regression in Nova [1] caused by a qemu bug [2] in the qemu version that is shipped with Focal.
The Nova team documented[4] it as a limitation without a known workaround in Nova. Also as far as I know Canonical does not support downgrading the qemu version so I don't see a viable workaround on Focal side either.
Is the Nova reno[4] enough from the TC perspective to go forward with the switch to Focal and with the Victoria release?
cheers, gibi
[1] https://bugs.launchpad.net/nova/+bug/1882521 [2] https://bugs.launchpad.net/qemu/+bug/1894804 [3] https://governance.openstack.org/tc/reference/runtimes/victoria.html [4] https://review.opendev.org/#/c/752654
On Fri, 2020-09-18 at 13:01 +0200, Balázs Gibizer wrote:
On Fri, Sep 18, 2020 at 12:20, Radosław Piliszek radoslaw.piliszek@gmail.com wrote:
Is there a version of qemu that works on Focal that deployment projects could use now?
We use the train ubuntu cloud archive to get an unaffected qemu version, which is now 4.0
that is what we are going to do for bionic. we cant do that for focal. there is no focal package for train. the cloud archive can only be used for backprots of new packages form new releases.
we shoudl be abel to get 5.0 form the victoria cloud archive when 20.10 is released which hopefully wont have the issue we are seeing 4.3?
cheers, gibi
-yoctozepto
On Fri, Sep 18, 2020 at 12:16 PM Balázs Gibizer balazs.gibizer@est.tech wrote:
Hi,
I want to raise a potential conflict between the fact that OpenStack Victoria officially supports Ubuntu Focal[3] and a high severity regression in Nova [1] caused by a qemu bug [2] in the qemu version that is shipped with Focal.
The Nova team documented[4] it as a limitation without a known workaround in Nova. Also as far as I know Canonical does not support downgrading the qemu version so I don't see a viable workaround on Focal side either.
Is the Nova reno[4] enough from the TC perspective to go forward with the switch to Focal and with the Victoria release?
cheers, gibi
[1] https://bugs.launchpad.net/nova/+bug/1882521 [2] https://bugs.launchpad.net/qemu/+bug/1894804 [3] https://governance.openstack.org/tc/reference/runtimes/victoria.html [4] https://review.opendev.org/#/c/752654
On 18-09-20 13:01:07, Balázs Gibizer wrote:
On Fri, Sep 18, 2020 at 12:20, Radosław Piliszek radoslaw.piliszek@gmail.com wrote:
Is there a version of qemu that works on Focal that deployment projects could use now?
Not that I'm aware of, I don't believe Canonical have a UCA for Focal yet, let alone one that has a version of QEMU that doesn't hit this issue.
It's worth noting that we haven't been able to reproduce this outside of OpenStack CI *and* similar sized nested test environments running the tox -e full Tempest target on Focal or Bionic with the Ussuri UCA.
I've personally been unable to reproduce this on Fedora 32 with or without the virt-preview repo.
We use the train ubuntu cloud archive to get an unaffected qemu version, which is now 4.0
We are proposing doing this on our current Bionic based CI hosts so we can still go ahead with a planned min version bump in V:
https://review.opendev.org/#/q/status:open+topic:bump-libvirt-qemu-victoria
On Fri, Sep 18, 2020 at 12:16 PM Balázs Gibizer balazs.gibizer@est.tech wrote:
Hi,
I want to raise a potential conflict between the fact that OpenStack Victoria officially supports Ubuntu Focal[3] and a high severity regression in Nova [1] caused by a qemu bug [2] in the qemu version that is shipped with Focal.
The Nova team documented[4] it as a limitation without a known workaround in Nova. Also as far as I know Canonical does not support downgrading the qemu version so I don't see a viable workaround on Focal side either.
Is the Nova reno[4] enough from the TC perspective to go forward with the switch to Focal and with the Victoria release?
cheers, gibi
[1] https://bugs.launchpad.net/nova/+bug/1882521 [2] https://bugs.launchpad.net/qemu/+bug/1894804 [3] https://governance.openstack.org/tc/reference/runtimes/victoria.html [4] https://review.opendev.org/#/c/752654
On Fri, 2020-09-18 at 12:20 +0200, Radosław Piliszek wrote:
Is there a version of qemu that works on Focal that deployment projects could use now?
not that we are aware of. there are some PPAs we coudl try like https://launchpad.net/~jacob/+archive/ubuntu/virtualisation it looks like that is proving 5.0 which will be what ships in 20.10 so the victora cloud archive shoudl be updated with that eventually currenlty it does not have libvirt/qemu in the repo but it should once 20.10 is out. https://ubuntu-cloud.archive.canonical.com/ubuntu/dists/focal-proposed/victo... thats our best bet to resolve this short term on focal i think.
-yoctozepto
On Fri, Sep 18, 2020 at 12:16 PM Balázs Gibizer balazs.gibizer@est.tech wrote:
Hi,
I want to raise a potential conflict between the fact that OpenStack Victoria officially supports Ubuntu Focal[3] and a high severity regression in Nova [1] caused by a qemu bug [2] in the qemu version that is shipped with Focal.
The Nova team documented[4] it as a limitation without a known workaround in Nova. Also as far as I know Canonical does not support downgrading the qemu version so I don't see a viable workaround on Focal side either.
Is the Nova reno[4] enough from the TC perspective to go forward with the switch to Focal and with the Victoria release?
cheers, gibi
[1] https://bugs.launchpad.net/nova/+bug/1882521 [2] https://bugs.launchpad.net/qemu/+bug/1894804 [3] https://governance.openstack.org/tc/reference/runtimes/victoria.html [4] https://review.opendev.org/#/c/752654
---- On Fri, 18 Sep 2020 05:08:22 -0500 Balázs_Gibizer_ balazs.gibizer@est.tech wrote ----
Hi,
I want to raise a potential conflict between the fact that OpenStack Victoria officially supports Ubuntu Focal[3] and a high severity regression in Nova [1] caused by a qemu bug [2] in the qemu version that is shipped with Focal.
The Nova team documented[4] it as a limitation without a known workaround in Nova. Also as far as I know Canonical does not support downgrading the qemu version so I don't see a viable workaround on Focal side either.
Is the Nova reno[4] enough from the TC perspective to go forward with the switch to Focal and with the Victoria release?
IMO, that is what we can do in this situation. Also, we are going to keep running one job on Bionic so that we do cover the volume detach feature test coverage somewhere. And once qemu bug is fixed we can remove this limitation in W cycle.
-gmann
cheers, gibi
[1] https://bugs.launchpad.net/nova/+bug/1882521 [2] https://bugs.launchpad.net/qemu/+bug/1894804 [3] https://governance.openstack.org/tc/reference/runtimes/victoria.html [4] https://review.opendev.org/#/c/752654
participants (5)
-
Balázs Gibizer
-
Ghanshyam Mann
-
Lee Yarwood
-
Radosław Piliszek
-
Sean Mooney