Hello, The tempest-slow-py3 job has been failing consistently since 21 February[1]. Looking at the failure, there is one failing test which looks neutron related[2] (but might be tempest related, not sure). *tempest.scenario.test_network_basic_ops.TestNetworkBasicOps. test_subnet_details* File "/opt/stack/tempest/tempest/lib/common/ssh.py", line 238, in exec_command raise exceptions.SSHExecCommandFailed( tempest.lib.exceptions.SSHExecCommandFailed: Command 'set -eu -o pipefail; PATH=$PATH:/sbin:/usr/sbin; cat /var/run/udhcpc.eth0.pid', exit status: 1, stderr: cat: can't open '/var/run/udhcpc.eth0.pid': No such file or directory Cinder is currently in the feature freeze exception phase and we have few patches that need to be merged within this week (before 24 Feb), hence raising the issue on ML. [1] https://zuul.opendev.org/t/openstack/builds?job_name=tempest-slow-py3&project=openstack/cinder [2] https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/z... Thanks Rajat Dhasmana
Hi Rajat, Please take a look at https://review.opendev.org/c/openstack/devstack/+/874625. If it fixes your issue, it just got merged. Thanks & regards, Vishal Manchanda On Wed, Feb 22, 2023 at 3:23 PM Rajat Dhasmana <rdhasman@redhat.com> wrote:
Hello,
The tempest-slow-py3 job has been failing consistently since 21 February[1]. Looking at the failure, there is one failing test which looks neutron related[2] (but might be tempest related, not sure).
*tempest.scenario.test_network_basic_ops.TestNetworkBasicOps. test_subnet_details* File "/opt/stack/tempest/tempest/lib/common/ssh.py", line 238, in exec_command raise exceptions.SSHExecCommandFailed( tempest.lib.exceptions.SSHExecCommandFailed: Command 'set -eu -o pipefail; PATH=$PATH:/sbin:/usr/sbin; cat /var/run/udhcpc.eth0.pid', exit status: 1, stderr: cat: can't open '/var/run/udhcpc.eth0.pid': No such file or directory
Cinder is currently in the feature freeze exception phase and we have few patches that need to be merged within this week (before 24 Feb), hence raising the issue on ML.
[1] https://zuul.opendev.org/t/openstack/builds?job_name=tempest-slow-py3&project=openstack/cinder [2] https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/z...
Thanks Rajat Dhasmana
Hi, Dnia środa, 22 lutego 2023 11:05:36 CET vishal manchanda pisze:
Hi Rajat,
Please take a look at https://review.opendev.org/c/openstack/devstack/+/874625. If it fixes your issue, it just got merged.
Some time ago I added support for dhcpcd in tempest https://review.opendev.org/c/openstack/tempest/+/871270 so with that and change like https://review.opendev.org/c/openstack/neutron/+/871272 in job's configuration it should works fine with Cirros 0.6.1
Thanks & regards, Vishal Manchanda
On Wed, Feb 22, 2023 at 3:23 PM Rajat Dhasmana <rdhasman@redhat.com> wrote:
Hello,
The tempest-slow-py3 job has been failing consistently since 21 February[1]. Looking at the failure, there is one failing test which looks neutron related[2] (but might be tempest related, not sure).
*tempest.scenario.test_network_basic_ops.TestNetworkBasicOps. test_subnet_details* File "/opt/stack/tempest/tempest/lib/common/ssh.py", line 238, in exec_command raise exceptions.SSHExecCommandFailed( tempest.lib.exceptions.SSHExecCommandFailed: Command 'set -eu -o pipefail; PATH=$PATH:/sbin:/usr/sbin; cat /var/run/udhcpc.eth0.pid', exit status: 1, stderr: cat: can't open '/var/run/udhcpc.eth0.pid': No such file or directory
Cinder is currently in the feature freeze exception phase and we have few patches that need to be merged within this week (before 24 Feb), hence raising the issue on ML.
[1] https://zuul.opendev.org/t/openstack/builds?job_name=tempest-slow-py3&project=openstack/cinder [2] https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/z...
Thanks Rajat Dhasmana
-- Slawek Kaplonski Principal Software Engineer Red Hat
Some time ago I added support for dhcpcd in tempest https://review.opendev.org/c/openstack/tempest/+/871270
so with that and change like https://review.opendev.org/c/openstack/neutron/+/871272 in job's configuration it should works fine with Cirros 0.6.1
Thanks, Slawek, we're planning to use that. Unfortunately, we can't set that opt for every slow job (a job which runs the test which requires that opt to be set correctly). I tried that ( https://review.opendev.org/c/openstack/tempest/+/874586) before I realize there are many variations of jobs which run the test. Plus we can set that opt only for master, the older branches where the new cirros image won't be used can't be affected. That being said, our plan is to address that in Devstack and basically set that opt automatically when cirros > 0.6.0 is used. -- Martin Kopec Senior Software Quality Engineer Red Hat EMEA IM: kopecmartin
On Wed, Feb 22, 2023, at 7:55 AM, Martin Kopec wrote:
Some time ago I added support for dhcpcd in tempest https://review.opendev.org/c/openstack/tempest/+/871270
so with that and change like https://review.opendev.org/c/openstack/neutron/+/871272 in job's configuration it should works fine with Cirros 0.6.1
Thanks, Slawek, we're planning to use that. Unfortunately, we can't set that opt for every slow job (a job which runs the test which requires that opt to be set correctly). I tried that (https://review.opendev.org/c/openstack/tempest/+/874586) before I realize there are many variations of jobs which run the test. Plus we can set that opt only for master, the older branches where the new cirros image won't be used can't be affected. That being said, our plan is to address that in Devstack and basically set that opt automatically when cirros > 0.6.0 is used.
Why not fix this in tempest by having the tooling check which dhcp client is present on the image first? Then you don't need to edit configuration when you change images, it can just detect which command to run. Is the issue that the emulated VM is too slow to run extra commands like that?
-- Martin Kopec Senior Software Quality Engineer Red Hat EMEA IM: kopecmartin
---- On Wed, 22 Feb 2023 07:55:06 -0800 Martin Kopec wrote ---
Some time ago I added support for dhcpcd in tempest https://review.opendev.org/c/openstack/tempest/+/871270 so with that and change like https://review.opendev.org/c/openstack/neutron/+/871272 in job's configuration it should works fine with Cirros 0.6.1
Thanks, Slawek, we're planning to use that. Unfortunately, we can't set that opt for every slow job (a job which runs the test which requires that opt to be set correctly). I tried that (https://review.opendev.org/c/openstack/tempest/+/874586) before I realize there are many variations of jobs which run the test. Plus we can set that opt only for master, the older branches where the new cirros image won't be used can't be affected. That being said, our plan is to address that in Devstack and basically set that opt automatically when cirros > 0.6.0 is used.
Yeah, and it is not just jobs we need to fix overall installation of devstack for local run also so fixing it via tempest config option and setting that in lib/tempest based on cirros version is a good way forward. But for now, cirros version bump change has been reverted and it is good to recheck the failing job - https://review.opendev.org/c/openstack/devstack/+/874625 -gmann
-- Martin Kopec Senior Software Quality Engineer Red Hat EMEAIM: kopecmartin
Thanks everyone for your replies, CI is passing now. On Wed, Feb 22, 2023 at 11:15 PM Ghanshyam Mann <gmann@ghanshyammann.com> wrote:
---- On Wed, 22 Feb 2023 07:55:06 -0800 Martin Kopec wrote ---
Some time ago I added support for dhcpcd in tempest
so with that and change like https://review.opendev.org/c/openstack/neutron/+/871272 in job's configuration it should works fine with Cirros 0.6.1
Thanks, Slawek, we're planning to use that. Unfortunately, we can't set that opt for every slow job (a job which runs the test which requires that opt to be set correctly). I tried that ( https://review.opendev.org/c/openstack/tempest/+/874586) before I realize
https://review.opendev.org/c/openstack/tempest/+/871270 there are many variations of jobs which run the test. Plus we can set that opt only for master, the older branches where the new cirros image won't be used can't be affected. That being said, our plan is to address that in Devstack and basically set that opt automatically when cirros > 0.6.0 is used.
Yeah, and it is not just jobs we need to fix overall installation of devstack for local run also so fixing it via tempest config option and setting that in lib/tempest based on cirros version is a good way forward.
But for now, cirros version bump change has been reverted and it is good to recheck the failing job - https://review.opendev.org/c/openstack/devstack/+/874625
-gmann
-- Martin Kopec Senior Software Quality Engineer Red Hat EMEAIM: kopecmartin
participants (6)
-
Clark Boylan
-
Ghanshyam Mann
-
Martin Kopec
-
Rajat Dhasmana
-
Slawek Kaplonski
-
vishal manchanda