Open Stack

In the past 2 days the Jenkins failure rate for the Tempest testsuite
seems to have become incredibly high - seemingly over 50% failure rate.
While we can trigger a rerun using 'recheck', this causes delays when
you have run recheck multiple times for many patches.

Does anyone know what's going on with Tempest right now & more importantly
if there's a way to fix it ?

eg this patch

  https://review.openstack.org/#/c/19116/

And this result:

  http://logs.openstack.org/19116/5/check/gate-tempest-devstack-vm/25818/console.html.gz

12:44:47 ======================================================================
12:44:47 ERROR: test_reboot_server_hard (tempest.tests.compute.servers.test_server_actions.ServerActionsTestJSON)
12:44:47 ----------------------------------------------------------------------
12:44:47 Traceback (most recent call last):
12:44:47   File "/opt/stack/new/tempest/tempest/tests/compute/servers/test_server_actions.py", line 234, in setUp
12:44:47     ServerActionsTestBase.setUp(self)
12:44:47   File "/opt/stack/new/tempest/tempest/tests/compute/servers/test_server_actions.py", line 44, in setUp
12:44:47     self.client.wait_for_server_status(self.server_id, 'ACTIVE')
12:44:47   File "/opt/stack/new/tempest/tempest/services/compute/json/servers_client.py", line 156, in wait_for_server_status
12:44:47     raise exceptions.BuildErrorException(server_id=server_id)
12:44:47 BuildErrorException: Server fc9009c1-dafe-49b1-b01f-c575cee838f9 failed to build and is in ERROR status
12:44:47 
12:44:47 ----------------------------------------------------------------------
12:44:47 XML: nosetests-smoke.xml
12:44:47 ----------------------------------------------------------------------
12:44:47 Ran 84 tests in 837.230s
12:44:47 
12:44:47 FAILED (SKIP=15, errors=1)


Or any of these ones had the same errors

https://review.openstack.org/#/c/19381/
https://review.openstack.org/#/c/19301/
https://review.openstack.org/#/c/19304/
...

Regrds,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|