Open Stack

Hi,

I checked the recent neutron log and found the same error in
neutron-ovn-tempest-slow job in the check queue [1] of a neutron
change merged a couple of hours ago [0].
I don't see such errors in other changes merged recently (801068 and 801076).
779310 is a child commit of 776701, so it hit the same error, but
these are not directly related.
I believe reverting
https://review.opendev.org/c/openstack/neutron/+/776701 will fix the
issue.
I will update https://review.opendev.org/c/openstack/neutron/+/801478.

The reason this issue happened is a configuration mismatch between
tempest-slow/full used globally (in most projects) and
tempest-slow/full used in neutron gate.
The neutron gate has two types of the tempest-slow jobs:
neutron-ovs-tempest-slow and neutron-ovn-tempest-slow.
The former is voting but the latter is non-voting.
tempest-slow used globally is same as the latter
(neutron-ovn-tempest-slow) from the point of view of devstack
configuration.
This is the reason the neutron gate could not detect it before merging
the commit which triggered the error.

[0] https://review.opendev.org/c/openstack/neutron/+/776701
[1] https://zuul.opendev.org/t/openstack/build/5344d9bdec9346738499114c10a81aff/log/controller/logs/screen-q-svc.txt#1993

Thanks,
Akihiro Motoki (amotoki)

On Wed, Jul 21, 2021 at 7:04 AM Ghanshyam Mann <gmann at ghanshyammann.com> wrote:
>
> Hello Everyone,
>
> Since 2-3 hrs before, tempest-full-py3, tempest-slow-py3 and few more jobs started failing
> consistently in create_neutron_initial_network() method:
>
> "++ lib/neutron_plugins/services/l3:create_neutron_initial_network:214 :   oscwrap --os-cloud devstack-admin --os-region RegionOne network create --project a6b71b541987471595bf5e38f5fbe264 private"
>
> - https://zuul.openstack.org/builds?job_name=tempest-full-py3
> - https://zuul.openstack.org/builds?job_name=tempest-slow-py3
>
> Strange is that 'tempest-integrated-storage'|'compute' jobs are passing even the configuration is almost the same.
>
> Slaweq reported the below bug and trying some neutron revert to know the root cause. He will check
> it tomorrow morning.
>
> - https://bugs.launchpad.net/neutron/+bug/1936983
>
> Until we find the root cause/fix, do not recheck on these failures.
>
> -gmann
>