[Openstack] unit and integration tests results for Gerrit

Dan Prince dan.prince at rackspace.com
Tue Dec 13 15:26:42 UTC 2011

Hi Jay,

The tests should not be running concurrently.
We currently have 4 Natty Cloud Servers configured as unit test workers. The machines are shared between nova/glance/keystone.
The unit test runner is pretty simple:
Would using a unique FAKE_FILESYSTEM_ROOTDIR for each test run or test help out here? Perhaps prefixing each one w/ /tmp/glance-tests so that we could also add an explicit tmp dir cleanup command in the test suite runners as well.
In the meantime I'm happy to add an explicit cleanup for /tmp/glance-tests to make sure we have a clean slate for each test run.
-----Original Message-----
From: "Jay Pipes" <jaypipes at gmail.com>
Sent: Tuesday, December 13, 2011 7:14am
To: "Dan Prince" <dan.prince at rackspace.com>
Cc: "Openstack" <openstack at lists.launchpad.net>
Subject: Re: [Openstack] unit and integration tests results for Gerrit

Thanks Dan, looks like a great start on this. Not sure what's going on
with the unit test runs in Glance, though... As an example, see:


There are dozens of errors like this:

ERROR: test_add_member (glance.tests.unit.test_api.TestGlanceAPI)
Traceback (most recent call last):
 File "/tmp/tmp.ZaWdEoPfhP/glance_source/glance/tests/unit/test_api.py",
line 1937, in setUp
 File "/tmp/tmp.ZaWdEoPfhP/glance_source/glance/tests/stubs.py", line
63, in stub_out_filesystem_backend
OSError: [Errno 17] File exists: '/tmp/glance-tests'

That FAKE_FILESYSTEM_ROOTDIR is cleaned up in the tearDown() method of
API unit tests, and these tests run perfectly fine for me locally. I
was thinking that one of the following things may be occurring:

* The unlink() of the FAKE_FILESYSTEM_ROOTDIR is not fsync'ing fast
enough, resulting in setUp()'s call to create the /tmp/glance-tests
directory is stumbling over itself
* The tests are being run in parallel somehow?

I've seen the fsync behaviour cause havoc in some of the image cache
tests before, and the solution ended up putting a small wait loop in
the test code to wait until disk buffers were flushed and a cache file
was fully removed from the filesystem. That may be happening here?
These are being run on Cloud Servers, right?

/me just trying to figure out why tests would run differently in
smokestack than Jenkins or locally...

Thanks in advance for any insight.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20111213/db3d86a3/attachment.html>

More information about the Openstack mailing list