Open Stack

 ---- On Wed, 21 Jul 2021 01:53:59 -0500 Slawek Kaplonski <skaplons at redhat.com> wrote ----
 > Hi,
 > 
 > On środa, 21 lipca 2021 01:30:25 CEST Akihiro Motoki wrote:
 > > Hi,
 > > 
 > > I checked the recent neutron log and found the same error in
 > > neutron-ovn-tempest-slow job in the check queue [1] of a neutron
 > > change merged a couple of hours ago [0].
 > > I don't see such errors in other changes merged recently (801068 and 
 > 801076).
 > > 779310 is a child commit of 776701, so it hit the same error, but
 > > these are not directly related.
 > > I believe reverting
 > > https://review.opendev.org/c/openstack/neutron/+/776701 will fix the
 > > issue.
 > > I will update https://review.opendev.org/c/openstack/neutron/+/801478.
 > > 
 > > The reason this issue happened is a configuration mismatch between
 > > tempest-slow/full used globally (in most projects) and
 > > tempest-slow/full used in neutron gate.
 > > The neutron gate has two types of the tempest-slow jobs:
 > > neutron-ovs-tempest-slow and neutron-ovn-tempest-slow.
 > > The former is voting but the latter is non-voting.
 > > tempest-slow used globally is same as the latter
 > > (neutron-ovn-tempest-slow) from the point of view of devstack
 > > configuration.
 > > This is the reason the neutron gate could not detect it before merging
 > > the commit which triggered the error.
 > 
 > Thx for analysis and fast approval of that revert.
 > I just proposed  https://review.opendev.org/c/openstack/neutron/+/801598 to 
 > make that neutron job voting and gating and to avoid such issues in the 
 > future.

+1, having the ovn job which is the default in devstack as voting will help.

-gmann

 > 
 > > 
 > > [0] https://review.opendev.org/c/openstack/neutron/+/776701
 > > [1]
 > > https://zuul.opendev.org/t/openstack/build/5344d9bdec9346738499114c10a81aff/l
 > > og/controller/logs/screen-q-svc.txt#1993
 > > 
 > > Thanks,
 > > Akihiro Motoki (amotoki)
 > > 
 > > On Wed, Jul 21, 2021 at 7:04 AM Ghanshyam Mann <gmann at ghanshyammann.com> 
 > wrote:
 > > > Hello Everyone,
 > > > 
 > > > Since 2-3 hrs before, tempest-full-py3, tempest-slow-py3 and few more jobs
 > > > started failing consistently in create_neutron_initial_network() method:
 > > > 
 > > > "++ lib/neutron_plugins/services/l3:create_neutron_initial_network:214 :  
 > > > oscwrap --os-cloud devstack-admin --os-region RegionOne network create
 > > > --project a6b71b541987471595bf5e38f5fbe264 private"
 > > > 
 > > > - https://zuul.openstack.org/builds?job_name=tempest-full-py3
 > > > - https://zuul.openstack.org/builds?job_name=tempest-slow-py3
 > > > 
 > > > Strange is that 'tempest-integrated-storage'|'compute' jobs are passing 
 > even
 > > > the configuration is almost the same.
 > > > 
 > > > Slaweq reported the below bug and trying some neutron revert to know the
 > > > root cause. He will check it tomorrow morning.
 > > > 
 > > > - https://bugs.launchpad.net/neutron/+bug/1936983
 > > > 
 > > > Until we find the root cause/fix, do not recheck on these failures.
 > > > 
 > > > -gmann
 > 
 > 
 > -- 
 > Slawek Kaplonski
 > Principal Software Engineer
 > Red Hat