Open Stack

Le ven. 2 déc. 2022 à 17:26, Sofia Enriquez <senrique at redhat.com> a écrit :

> Hi,
> Could this problem affect cinder-tempest-plugin-lvm-tgt-barbican jobs as
> well? (All the nova+lvm test are failing)
> https://zuul.opendev.org/t/openstack/build/140630488ea745a69ce3ebadf85a41fa
>
>

Sounds related AFAICS as this is related to a volume attach/detach.
TBC, the root problem is that when you attach or detach a volume earlier
than when the guest kernel is fully booted, then detach won't work.
In order to not have a volume detach problem, Tempest needs to reliably
wait for the guest to be sshable (as the kernel will be booted).

HTH,
-Sylvain

Thanks
> Sofia
>
> On Tue, Nov 29, 2022 at 3:22 PM Sylvain Bauza <sbauza at redhat.com> wrote:
>
>>
>>
>> Le mar. 29 nov. 2022 à 09:33, Sylvain Bauza <sbauza at redhat.com> a écrit :
>>
>>> (early morning, needing a coffee apparently)
>>>
>>> Le mar. 29 nov. 2022 à 09:32, Sylvain Bauza <sbauza at redhat.com> a
>>> écrit :
>>>
>>>>
>>>>
>>>> Le lun. 28 nov. 2022 à 14:28, Sylvain Bauza <sbauza at redhat.com> a
>>>> écrit :
>>>>
>>>>> Sorry folks, that's kind of an email I hate writing but let's be
>>>>> honest : our gate is busted.
>>>>> Until we figure out a correct path for resolution, I hereby ask you to
>>>>> *NOT* recheck in order to not spill our precious CI resources for tests
>>>>> that are certain to fail.
>>>>>
>>>>> Long story story, there are currently two problems :
>>>>> #1 https://launchpad.net/bugs/1940425 nova-ovs-hybrid-plug and
>>>>> nova-next jobs 100% fail due to a port remaining in down state.
>>>>> #2 https://bugs.launchpad.net/nova/+bug/1960346 nova-lvm job 100%
>>>>> fails due to a volume detach failure probably due to QEMU
>>>>>
>>>>>
>>>>>
>>>> Today's update :
>>>>
>>>>
>>>>> #1 is currently investigated by the Neutron team meanwhile a patch [1]
>>>>> has been proposed against Zuul to skip the failing tests.
>>>>> Unfortunately, this patch [1] is unable to merge due to #2.
>>>>>
>>>>>
>>>> Good news, kudos to the Neutron team which delivered a bugfix against
>>>> the rootcause, which is always better than just skipping tests (and lacking
>>>> then coverage).
>>>> https://review.opendev.org/c/openstack/neutron/+/837780/18
>>>>
>>>> Accordingly, [1] is no longer necessary and has been abandoned after a
>>>> recheck to verify the job runs.
>>>>
>>>>
>>>>
>>>>> #2 has a Tempest patch that's being worked on [2] but the current
>>>>> state of this patch is WIP.
>>>>> We somehow need to have an agreement on the way forward during this
>>>>> afternoon (UTC) to identify whether we can reasonably progress on [2] or
>>>>> skip the failing tests on nova-lvm.
>>>>>
>>>>>
>>>> Given [2] is hard to write, gmann proposed a patch [3] for skipping
>>>> some nova-lvm tests. Reviews of [3] ongoing, should be hopefully merged
>>>> today around noon UTC.
>>>>
>>>> Once [3] is merged, the gate should be unblocked.
>>>> Again, an email will be sent once we progress on [3].
>>>>
>>>
>> [3] is merged, so now the gate is back \o/
>> Thanks all folks who helped on those issues !
>>
>>
>>> -S
>>>>
>>>>
>>>>> Again, sorry about the bad news and I'll keep you informed.
>>>>> -Sylvain
>>>>>
>>>>> [1] https://review.opendev.org/c/openstack/nova/+/865658/
>>>>> [2] https://review.opendev.org/c/openstack/tempest/+/842240
>>>>>
>>>> [3] https://review.opendev.org/c/openstack/nova/+/865922
>>>
>>
>
> --
>
> Sofía Enriquez
>
> she/her
>
> Software Engineer
>
> Red Hat PnT <https://www.redhat.com>
>
> IRC: @enriquetaso
> @RedHat <https://twitter.com/redhat>   Red Hat
> <https://www.linkedin.com/company/red-hat>  Red Hat
> <https://www.facebook.com/RedHatInc>
> <https://www.redhat.com>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20221205/148fbfcd/attachment.htm>