Hi, I checked the recent neutron log and found the same error in neutron-ovn-tempest-slow job in the check queue [1] of a neutron change merged a couple of hours ago [0]. I don't see such errors in other changes merged recently (801068 and 801076). 779310 is a child commit of 776701, so it hit the same error, but these are not directly related. I believe reverting https://review.opendev.org/c/openstack/neutron/+/776701 will fix the issue. I will update https://review.opendev.org/c/openstack/neutron/+/801478. The reason this issue happened is a configuration mismatch between tempest-slow/full used globally (in most projects) and tempest-slow/full used in neutron gate. The neutron gate has two types of the tempest-slow jobs: neutron-ovs-tempest-slow and neutron-ovn-tempest-slow. The former is voting but the latter is non-voting. tempest-slow used globally is same as the latter (neutron-ovn-tempest-slow) from the point of view of devstack configuration. This is the reason the neutron gate could not detect it before merging the commit which triggered the error. [0] https://review.opendev.org/c/openstack/neutron/+/776701 [1] https://zuul.opendev.org/t/openstack/build/5344d9bdec9346738499114c10a81aff/... Thanks, Akihiro Motoki (amotoki) On Wed, Jul 21, 2021 at 7:04 AM Ghanshyam Mann <gmann@ghanshyammann.com> wrote:
Hello Everyone,
Since 2-3 hrs before, tempest-full-py3, tempest-slow-py3 and few more jobs started failing consistently in create_neutron_initial_network() method:
"++ lib/neutron_plugins/services/l3:create_neutron_initial_network:214 : oscwrap --os-cloud devstack-admin --os-region RegionOne network create --project a6b71b541987471595bf5e38f5fbe264 private"
- https://zuul.openstack.org/builds?job_name=tempest-full-py3 - https://zuul.openstack.org/builds?job_name=tempest-slow-py3
Strange is that 'tempest-integrated-storage'|'compute' jobs are passing even the configuration is almost the same.
Slaweq reported the below bug and trying some neutron revert to know the root cause. He will check it tomorrow morning.
- https://bugs.launchpad.net/neutron/+bug/1936983
Until we find the root cause/fix, do not recheck on these failures.
-gmann