On Tue, 20 Jul 2021, 16:59 -0500, Ghanshyam Mann <gmann@ghanshyammann.com> wrote:
Hello Everyone,
Since 2-3 hrs before, tempest-full-py3, tempest-slow-py3 and few more jobs started failing consistently in create_neutron_initial_network() method:
"++ lib/neutron_plugins/services/l3:create_neutron_initial_network:214 : oscwrap --os-cloud devstack-admin --os-region RegionOne network create --project a6b71b541987471595bf5e38f5fbe264 private"
- https://zuul.openstack.org/builds?job_name=tempest-full-py3 - https://zuul.openstack.org/builds?job_name=tempest-slow-py3
Strange is that 'tempest-integrated-storage'|'compute' jobs are passing even the configuration is almost the same.
Slaweq reported the below bug and trying some neutron revert to know the root cause. He will check it tomorrow morning.
- https://bugs.launchpad.net/neutron/+bug/1936983
Until we find the root cause/fix, do not recheck on these failures.
FYI all: nova-multi-cell, nova-live-migration, and nova-ceph-multistore are also affected: https://zuul.opendev.org/t/openstack/builds?job_name=nova-multi-cell https://zuul.opendev.org/t/openstack/builds?job_name=nova-live-migration https://zuul.opendev.org/t/openstack/builds?job_name=nova-ceph-multistore And here's what the error trace looks like, for anyone who hasn't seen it yet:
Error while executing command: HttpException: 500, Request Failed: internal server error while processing your request. ++ functions-common:oscwrap:2349 : return 1 + lib/neutron_plugins/services/l3:create_neutron_initial_network:214 : NET_ID= + lib/neutron_plugins/services/l3:create_neutron_initial_network:215 : die_if_not_set 215 NET_ID 'Failure creating NET_ID for private 0b1d94f08f194eb5b7679c47f91f6d4c' + functions-common:die_if_not_set:216 : local exitcode=0 [Call Trace] ./stack.sh:1300:create_neutron_initial_network /opt/stack/devstack/lib/neutron_plugins/services/l3:215:die_if_not_set /opt/stack/devstack/functions-common:223:die [ERROR] /opt/stack/devstack/functions-common:215 Failure creating NET_ID for private 0b1d94f08f194eb5b7679c47f91f6d4c exit_trap: cleaning up child processes Error on exit *** FINISHED ***
-melwitt