[openstack-dev] Building on Debian: Havana unit tests at build time report

Thomas Goirand zigo at debian.org
Thu Oct 17 07:16:22 UTC 2013


Hi,

As you know, I maintain OpenStack both for Debian Sid, and a backport
for Wheezy (non-official) repository (since using the backports
repository for OpenStack wouldn't be practical). Below is the result of
the building OpenStack Havana (latest RC) under both these environments.

I would really appreciate if people from each of the concerned projects
could pay attention to this report, and provide fixes. It doesn't have
to be merged into the final release (I of course understand that we are
running out of time), though if there's a proposed patch, even if it's
not yet merged and still with "work in progress" status, that's fine for
me. I can use in a Debian specific patch (under debian/patches), and
upload later on.

Another thing which struck me: OpenStack really works well in Sid, even
with SQLAlchemy 0.8.2, apart from this Nova glitch which *must* be
fixed. I really hope someone will come with a patch for it.

I'd like also to point out that if I am publishing this report, it
really is because I expect others to work on it. Remember that I'm alone
doing the Debian packaging work (as opposed to a full team for Ubuntu),
and that you cannot expect that *I* fix (all) things. I simply don't
have enough time to work on it myself, unfortunately. So help wanted... :)

Last thing: please test and use Debian (Sid or my Wheezy backport
repository) for OpenStack!

Cheers,

Thomas Goirand (zigo)

P.S: If you guys like this kind of reports, I can post this more often
in the list. I welcome any comment on improving the form of it. Also
remember that my jenkins build server for Wheezy is public, and that you
can join #debian-openstack-commits to see the real-time status of
package builds, plus #debian-openstack to discuss (on the
irc.debian.org, aka OFTC network, not on Freenode).

Here's the report.

* Ceilometer
============

No failure, except for calling binaries which are not installed (yet) at
package build time, called by subprocess.py. Example:

FAIL: tests.test_bin.BinTestCase.test_run_expirer
tags: worker-0
----------------------------------------------------------------------
Empty attachments:
  pythonlogging:''

Traceback (most recent call last):
  File
"/home/zigo/sources/openstack/havana/ceilometer/build-area/ceilometer-2013.2~rc2/tests/test_bin.py",
line 45, in test_run_expirer
    "--config-file=%s" % self.tempfile])
  File "/usr/lib/python2.7/subprocess.py", line 709, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1326, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Same result under Wheezy backports and Sid.

In Quantum/Neutron, this has been fixed, but I can't remember how. Could
anyone tell? It'd be great if it could be fixed.

Note that I already uploaded python-wsme 0.5b6 which is needed by the
last RC of Ceilometer, so we are fine on this ground.

* Cinder
========

No unit test failure / error (in both Sid and Wheezy). \o/

* Glance
========

No unit test failure / error (in both Sid and Wheezy). \o/

* Heat
======

No unit test failure / error (in both Sid and Wheezy). However, Heat
doesn't include a -P option in ./run_tests.sh, and insists on running
the PEP8 checks, which fails because Sid has pyflakes 0.7.3, and Heat
wants 0.7.2:

pkg_resources.VersionConflict: (pyflakes 0.7.3
(/usr/lib/python2.7/dist-packages), Requirement.parse('pyflakes==0.7.2'))

I had the same problem with Sid, with the PEP8 1.4.4 which I maintain as
backport (and Heat expects 1.4.5). I could work it out, though I don't
think it's worse the trouble (as it will not change the useability of
the package), so I just gave up on it.

Also, heat uses /usr/bin/coverage. Please switch to python -m coverage
(IMO, the best way...), or check if /usr/bin/coverage and if not, use
/usr/bin/python-coverage.

* Horizon
=========

Under Sid, sometimes the unit tests are passing without an issue, and
*sometimes* there's a timeout with Selenium that produces errors:

ERROR: test suite for <class
'openstack_dashboard.dashboards.admin.users.tests.SeleniumTests'>
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/suite.py", line 227, in run
    self.tearDown()
  File "/usr/lib/python2.7/dist-packages/nose/suite.py", line 350, in
tearDown
    self.teardownContext(ancestor)
  File "/usr/lib/python2.7/dist-packages/nose/suite.py", line 366, in
teardownContext
    try_run(context, names)
  File "/usr/lib/python2.7/dist-packages/nose/util.py", line 469, in try_run
    return func()
  File
"/home/zigo/sources/openstack/havana/horizon/build-area/horizon-2013.2~rc3/horizon/test/helpers.py",
line 179, in tearDownClass
    super(SeleniumTestCase, cls).tearDownClass()
  File "/usr/lib/python2.7/dist-packages/django/test/testcases.py", line
1170, in tearDownClass
    cls.server_thread.join()
  File "/usr/lib/python2.7/dist-packages/django/test/testcases.py", line
1094, in join
    self.httpd.shutdown()
  File "/usr/lib/python2.7/dist-packages/django/test/testcases.py", line
984, in shutdown
    "Failed to shutdown the live test server in 2 seconds. The "
RuntimeError: Failed to shutdown the live test server in 2 seconds. The
server might be stuck or generating a slow response.

This is surprising, because the python-selenium, which is non-free,
isn't installed in my environment, and we were supposed to have a patch
to not use it if it was detected it wasn't there. The same way, there's
this one:

ERROR: test suite for <class
'openstack_dashboard.test.tests.selenium.BrowserTests'>
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/suite.py", line 208, in run
    self.setUp()
  File "/usr/lib/python2.7/dist-packages/nose/suite.py", line 291, in setUp
    self.setupContext(ancestor)
  File "/usr/lib/python2.7/dist-packages/nose/suite.py", line 314, in
setupContext
    try_run(context, names)
  File "/usr/lib/python2.7/dist-packages/nose/util.py", line 469, in try_run
    return func()
  File
"/home/zigo/sources/openstack/havana/horizon/build-area/horizon-2013.2~rc3/horizon/test/helpers.py",
line 173, in setUpClass
    super(SeleniumTestCase, cls).setUpClass()
  File "/usr/lib/python2.7/dist-packages/django/test/testcases.py", line
1160, in setUpClass
    raise cls.server_thread.error
WSGIServerException: [Errno 98] Address already in use

Also, Horizon insist on a requirement on python-coverage wrapper in
/usr/bin/coverage:

Running Horizon application tests
./run_tests.sh: line 301: coverage: command not found
./run_tests.sh: line 302: coverage: command not found
Running openstack_dashboard tests
./run_tests.sh: line 311: coverage: command not found
Tests failed.

As I wrote previously, the Debian package only has
/usr/bin/python-coverage, and the current maintainer refuses to modify
his package to provide /usr/bin/coverage. Instead, Horizon should be
using "python -m coverage" which will always work, so in fails on my
Jenkins cowbuilder environment. However, when I build on my Wheezy
laptop, with a manually added symlink for the coverage command, I
haven't see any unit test error.

* Keystone
==========

Under Sid, I have a single unit test failure:

FAIL:
keystone.tests.test_wsgi.LocalizedResponseTest.test_static_translated_string_is_Message
----------------------------------------------------------------------
_StringException: Traceback (most recent call last):
  File
"/home/zigo/sources/openstack/havana/keystone/build-area/keystone-2013.2~rc3/keystone/tests/test_wsgi.py",
line 306, in test_static_translated_string_is_Message
    gettextutils.Message)
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line
373, in assertIsInstance
    self.assertThat(obj, matcher, msg)
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line
417, in assertThat
    raise MismatchError(matchee, matcher, mismatch, verbose)
MismatchError: 'The request you have made requires authentication.' is
not an instance of Message

-------------------- >> begin captured logging << --------------------
keystone.common.cache.core: DEBUG: Adding cache-proxy
'keystone.tests.test_cache.CacheIsolatingProxy' to backend.
--------------------- >> end captured logging << ---------------------

And in Wheezy, what's below:

ERROR:
keystone.tests.test_wsgi.LocalizedResponseTest.test_static_translated_string_is_Message
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 133, in run
    self.runTest(result)
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 151, in runTest
    test(result)
  File "/usr/lib/python2.7/unittest/case.py", line 396, in __call__
    return self.run(*args, **kwds)
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line
526, in run
    return self.__RunTest(self, self.exception_handlers).run(result)
  File "/usr/lib/python2.7/dist-packages/testtools/runtest.py", line 74,
in run
    return self._run_one(actual_result)
  File "/usr/lib/python2.7/dist-packages/testtools/runtest.py", line 88,
in _run_one
    return self._run_prepared_result(ExtendedToOriginalDecorator(result))
  File "/usr/lib/python2.7/dist-packages/testtools/runtest.py", line
107, in _run_prepared_result
    handler(self.case, self.result, e)
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line
499, in _report_failure
    result.addFailure(self, details=self.getDetails())
  File "/usr/lib/python2.7/dist-packages/testtools/testresult/real.py",
line 1151, in addFailure
    return self.decorated.addFailure(test, err)
  File "/usr/lib/python2.7/dist-packages/nose/proxy.py", line 146, in
addFailure
    formatted = plugins.formatFailure(self.test, err)
  File "/usr/lib/python2.7/dist-packages/nose/plugins/manager.py", line
94, in __call__
    return self.call(*arg, **kw)
  File "/usr/lib/python2.7/dist-packages/nose/plugins/manager.py", line
136, in chain
    result = meth(*arg, **kw)
  File "/usr/lib/python2.7/dist-packages/nose/plugins/failuredetail.py",
line 41, in formatFailure
    tbinfo = inspect_traceback(tb)
  File "/usr/lib/python2.7/dist-packages/nose/inspector.py", line 26, in
inspect_traceback
    while tb.tb_next:
AttributeError: 'NoneType' object has no attribute 'tb_next'
-------------------- >> begin captured logging << --------------------
keystone.common.cache.core: DEBUG: Adding cache-proxy
'keystone.tests.test_cache.CacheIsolatingProxy' to backend.
--------------------- >> end captured logging << ---------------------

It surprised me that it's not the same error.

* Neutron
=========

Only 2 failures which don't seem so important under Sid:

FAIL: process-returncode
process-returncode
----------------------------------------------------------------------
_StringException: Binary content:
  traceback (test/plain; charset="utf8")


======================================================================
FAIL:
neutron.tests.unit.ml2.drivers.test_cisco_mech.TestCiscoBasicGet.test_single_get_admin
neutron.tests.unit.ml2.drivers.test_cisco_mech.TestCiscoBasicGet.test_single_get_admin
----------------------------------------------------------------------
_StringException


These appear in both Sid and Wheezy backports.

* Nova
======

Only a single: FAIL:
nova.tests.db.test_migrations.TestNovaMigrations.test_walk_versions in
Sid. The relevant part is:

OperationalError: (OperationalError) table instances has no column named
cleaned u'CREATE INDEX instances_host_deleted_cleaned_idx ON instances
(host, deleted, cleaned)' ()

It would really be nice if that one could be fixed, and that's the
failure which worries me the most. Note that this is using sqlite3.
There's no error at all under Wheezy backport, which leads me to believe
the error is linked to SQLAlchemy 0.8.2 in Sid.

Thanks for reading this report.



More information about the OpenStack-dev mailing list