Open Stack

On 10/17/2013 03:16 AM, Thomas Goirand wrote:
> Hi,
> 
> As you know, I maintain OpenStack both for Debian Sid, and a backport
> for Wheezy (non-official) repository (since using the backports
> repository for OpenStack wouldn't be practical). Below is the result of
> the building OpenStack Havana (latest RC) under both these environments.

Awesome - thanks! I agree with Robert, I think there are several bug
reports in this list.

> I would really appreciate if people from each of the concerned projects
> could pay attention to this report, and provide fixes. It doesn't have
> to be merged into the final release (I of course understand that we are
> running out of time), though if there's a proposed patch, even if it's
> not yet merged and still with "work in progress" status, that's fine for
> me. I can use in a Debian specific patch (under debian/patches), and
> upload later on.
> 
> Another thing which struck me: OpenStack really works well in Sid, even
> with SQLAlchemy 0.8.2, apart from this Nova glitch which *must* be
> fixed. I really hope someone will come with a patch for it.

I am in favor of us figuring out the 0.8 upgrade early in the icehouse
cycle.

> I'd like also to point out that if I am publishing this report, it
> really is because I expect others to work on it. Remember that I'm alone
> doing the Debian packaging work (as opposed to a full team for Ubuntu),
> and that you cannot expect that *I* fix (all) things. I simply don't
> have enough time to work on it myself, unfortunately. So help wanted... :)
> 
> Last thing: please test and use Debian (Sid or my Wheezy backport
> repository) for OpenStack!
> 
> Cheers,
> 
> Thomas Goirand (zigo)
> 
> P.S: If you guys like this kind of reports, I can post this more often
> in the list. I welcome any comment on improving the form of it. Also
> remember that my jenkins build server for Wheezy is public, and that you
> can join #debian-openstack-commits to see the real-time status of
> package builds, plus #debian-openstack to discuss (on the
> irc.debian.org, aka OFTC network, not on Freenode).
> 
> Here's the report.
> 
> * Ceilometer
> ============
> 
> No failure, except for calling binaries which are not installed (yet) at
> package build time, called by subprocess.py. Example:
> 
> FAIL: tests.test_bin.BinTestCase.test_run_expirer
> tags: worker-0
> ----------------------------------------------------------------------
> Empty attachments:
>   pythonlogging:''
> 
> Traceback (most recent call last):
>   File
> "/home/zigo/sources/openstack/havana/ceilometer/build-area/ceilometer-2013.2~rc2/tests/test_bin.py",
> line 45, in test_run_expirer
>     "--config-file=%s" % self.tempfile])
>   File "/usr/lib/python2.7/subprocess.py", line 709, in __init__
>     errread, errwrite)
>   File "/usr/lib/python2.7/subprocess.py", line 1326, in _execute_child
>     raise child_exception
> OSError: [Errno 2] No such file or directory
> 
> Same result under Wheezy backports and Sid.
> 
> In Quantum/Neutron, this has been fixed, but I can't remember how. Could
> anyone tell? It'd be great if it could be fixed.

Def should be a bug.

> Note that I already uploaded python-wsme 0.5b6 which is needed by the
> last RC of Ceilometer, so we are fine on this ground.

There is a bug here too. You say that Ceilometer needs 0.5b6 - but
global-requirements.txt only lists 0.5b5. We should fix that.

Additionally, I believe we need to stop depending on betas of things
that aren't participating in the gate.

> * Cinder
> ========
> 
> No unit test failure / error (in both Sid and Wheezy). \o/
> 
> * Glance
> ========
> 
> No unit test failure / error (in both Sid and Wheezy). \o/
> 
> * Heat
> ======
> 
> No unit test failure / error (in both Sid and Wheezy). However, Heat
> doesn't include a -P option in ./run_tests.sh, and insists on running
> the PEP8 checks, which fails because Sid has pyflakes 0.7.3, and Heat
> wants 0.7.2:

You should not be running run_tests.sh really. Try python setup.py test.
Or, in fact, just remove any test overrides you have in debian rules
files, as python setup.py test should work now and that should be what
dh calls anyway.

> pkg_resources.VersionConflict: (pyflakes 0.7.3
> (/usr/lib/python2.7/dist-packages), Requirement.parse('pyflakes==0.7.2'))

That sounds like a bug in heat. Its requirements list is out of sync.
hacking in its current release depends on >=0.7.2,<0.7.4.

> I had the same problem with Sid, with the PEP8 1.4.4 which I maintain as
> backport (and Heat expects 1.4.5). I could work it out, though I don't
> think it's worse the trouble (as it will not change the useability of
> the package), so I just gave up on it.

The pep8 pin for all of openstack is 1.4.5. Now that icehouse is open,
we might look at newer versions and consider whether there is value in
changing the pin.

However, python setup.py test and this issue should go away.

> Also, heat uses /usr/bin/coverage. Please switch to python -m coverage
> (IMO, the best way...), or check if /usr/bin/coverage and if not, use
> /usr/bin/python-coverage.

Coverage should be run via:

  python setup.py testr --coverage

run_tests.sh is a completely unsupported file that is there purely for
developer convenience.

> * Horizon
> =========
> 
> Under Sid, sometimes the unit tests are passing without an issue, and
> *sometimes* there's a timeout with Selenium that produces errors:
> 
> ERROR: test suite for <class
> 'openstack_dashboard.dashboards.admin.users.tests.SeleniumTests'>
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/dist-packages/nose/suite.py", line 227, in run
>     self.tearDown()
>   File "/usr/lib/python2.7/dist-packages/nose/suite.py", line 350, in
> tearDown
>     self.teardownContext(ancestor)
>   File "/usr/lib/python2.7/dist-packages/nose/suite.py", line 366, in
> teardownContext
>     try_run(context, names)
>   File "/usr/lib/python2.7/dist-packages/nose/util.py", line 469, in try_run
>     return func()
>   File
> "/home/zigo/sources/openstack/havana/horizon/build-area/horizon-2013.2~rc3/horizon/test/helpers.py",
> line 179, in tearDownClass
>     super(SeleniumTestCase, cls).tearDownClass()
>   File "/usr/lib/python2.7/dist-packages/django/test/testcases.py", line
> 1170, in tearDownClass
>     cls.server_thread.join()
>   File "/usr/lib/python2.7/dist-packages/django/test/testcases.py", line
> 1094, in join
>     self.httpd.shutdown()
>   File "/usr/lib/python2.7/dist-packages/django/test/testcases.py", line
> 984, in shutdown
>     "Failed to shutdown the live test server in 2 seconds. The "
> RuntimeError: Failed to shutdown the live test server in 2 seconds. The
> server might be stuck or generating a slow response.
> 
> This is surprising, because the python-selenium, which is non-free,
> isn't installed in my environment, and we were supposed to have a patch
> to not use it if it was detected it wasn't there. The same way, there's
> this one:
> 
> ERROR: test suite for <class
> 'openstack_dashboard.test.tests.selenium.BrowserTests'>
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/dist-packages/nose/suite.py", line 208, in run
>     self.setUp()
>   File "/usr/lib/python2.7/dist-packages/nose/suite.py", line 291, in setUp
>     self.setupContext(ancestor)
>   File "/usr/lib/python2.7/dist-packages/nose/suite.py", line 314, in
> setupContext
>     try_run(context, names)
>   File "/usr/lib/python2.7/dist-packages/nose/util.py", line 469, in try_run
>     return func()
>   File
> "/home/zigo/sources/openstack/havana/horizon/build-area/horizon-2013.2~rc3/horizon/test/helpers.py",
> line 173, in setUpClass
>     super(SeleniumTestCase, cls).setUpClass()
>   File "/usr/lib/python2.7/dist-packages/django/test/testcases.py", line
> 1160, in setUpClass
>     raise cls.server_thread.error
> WSGIServerException: [Errno 98] Address already in use

Sounds like a bug.

> Also, Horizon insist on a requirement on python-coverage wrapper in
> /usr/bin/coverage:
> 
> Running Horizon application tests
> ./run_tests.sh: line 301: coverage: command not found
> ./run_tests.sh: line 302: coverage: command not found
> Running openstack_dashboard tests
> ./run_tests.sh: line 311: coverage: command not found
> Tests failed.

run_tests.sh is not supported. However, I'm not 100% confident that
we've got horizon up to date on everything- so I think this could count
as a bug.

> As I wrote previously, the Debian package only has
> /usr/bin/python-coverage, and the current maintainer refuses to modify
> his package to provide /usr/bin/coverage. Instead, Horizon should be
> using "python -m coverage" which will always work, so in fails on my
> Jenkins cowbuilder environment. However, when I build on my Wheezy
> laptop, with a manually added symlink for the coverage command, I
> haven't see any unit test error.
> 
> * Keystone
> ==========
> 
> Under Sid, I have a single unit test failure:
> 
> FAIL:
> keystone.tests.test_wsgi.LocalizedResponseTest.test_static_translated_string_is_Message
> ----------------------------------------------------------------------
> _StringException: Traceback (most recent call last):
>   File
> "/home/zigo/sources/openstack/havana/keystone/build-area/keystone-2013.2~rc3/keystone/tests/test_wsgi.py",
> line 306, in test_static_translated_string_is_Message
>     gettextutils.Message)
>   File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line
> 373, in assertIsInstance
>     self.assertThat(obj, matcher, msg)
>   File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line
> 417, in assertThat
>     raise MismatchError(matchee, matcher, mismatch, verbose)
> MismatchError: 'The request you have made requires authentication.' is
> not an instance of Message
> 
> -------------------- >> begin captured logging << --------------------
> keystone.common.cache.core: DEBUG: Adding cache-proxy
> 'keystone.tests.test_cache.CacheIsolatingProxy' to backend.
> --------------------- >> end captured logging << ---------------------
> 
> And in Wheezy, what's below:
> 
> ERROR:
> keystone.tests.test_wsgi.LocalizedResponseTest.test_static_translated_string_is_Message
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/dist-packages/nose/case.py", line 133, in run
>     self.runTest(result)
>   File "/usr/lib/python2.7/dist-packages/nose/case.py", line 151, in runTest
>     test(result)
>   File "/usr/lib/python2.7/unittest/case.py", line 396, in __call__
>     return self.run(*args, **kwds)
>   File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line
> 526, in run
>     return self.__RunTest(self, self.exception_handlers).run(result)
>   File "/usr/lib/python2.7/dist-packages/testtools/runtest.py", line 74,
> in run
>     return self._run_one(actual_result)
>   File "/usr/lib/python2.7/dist-packages/testtools/runtest.py", line 88,
> in _run_one
>     return self._run_prepared_result(ExtendedToOriginalDecorator(result))
>   File "/usr/lib/python2.7/dist-packages/testtools/runtest.py", line
> 107, in _run_prepared_result
>     handler(self.case, self.result, e)
>   File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line
> 499, in _report_failure
>     result.addFailure(self, details=self.getDetails())
>   File "/usr/lib/python2.7/dist-packages/testtools/testresult/real.py",
> line 1151, in addFailure
>     return self.decorated.addFailure(test, err)
>   File "/usr/lib/python2.7/dist-packages/nose/proxy.py", line 146, in
> addFailure
>     formatted = plugins.formatFailure(self.test, err)
>   File "/usr/lib/python2.7/dist-packages/nose/plugins/manager.py", line
> 94, in __call__
>     return self.call(*arg, **kw)
>   File "/usr/lib/python2.7/dist-packages/nose/plugins/manager.py", line
> 136, in chain
>     result = meth(*arg, **kw)
>   File "/usr/lib/python2.7/dist-packages/nose/plugins/failuredetail.py",
> line 41, in formatFailure
>     tbinfo = inspect_traceback(tb)
>   File "/usr/lib/python2.7/dist-packages/nose/inspector.py", line 26, in
> inspect_traceback
>     while tb.tb_next:
> AttributeError: 'NoneType' object has no attribute 'tb_next'
> -------------------- >> begin captured logging << --------------------
> keystone.common.cache.core: DEBUG: Adding cache-proxy
> 'keystone.tests.test_cache.CacheIsolatingProxy' to backend.
> --------------------- >> end captured logging << ---------------------
> 
> It surprised me that it's not the same error.

Sounds lke maybe TWO bugs! :)

> * Neutron
> =========
> 
> Only 2 failures which don't seem so important under Sid:
> 
> FAIL: process-returncode
> process-returncode
> ----------------------------------------------------------------------
> _StringException: Binary content:
>   traceback (test/plain; charset="utf8")
> 
> 
> ======================================================================
> FAIL:
> neutron.tests.unit.ml2.drivers.test_cisco_mech.TestCiscoBasicGet.test_single_get_admin
> neutron.tests.unit.ml2.drivers.test_cisco_mech.TestCiscoBasicGet.test_single_get_admin
> ----------------------------------------------------------------------
> _StringException
> 
> 
> These appear in both Sid and Wheezy backports.
> 
> * Nova
> ======
> 
> Only a single: FAIL:
> nova.tests.db.test_migrations.TestNovaMigrations.test_walk_versions in
> Sid. The relevant part is:
> 
> OperationalError: (OperationalError) table instances has no column named
> cleaned u'CREATE INDEX instances_host_deleted_cleaned_idx ON instances
> (host, deleted, cleaned)' ()
> 
> It would really be nice if that one could be fixed, and that's the
> failure which worries me the most. Note that this is using sqlite3.
> There's no error at all under Wheezy backport, which leads me to believe
> the error is linked to SQLAlchemy 0.8.2 in Sid.

Yes. As Robert pointed out, we don't support that yet. BUT - as I said,
I highly support us upgrading to 0.8 now that icehouse is underway.
Perhaps a bug?