Open Stack

In car... But tl;dr there is another project with a similar problem. I'll see if I can make you a patch.

However, I believe the problem is actually the hacking import only modules test. We might want to revisit that one... It's hard to safely test, and even in nova it has exceptions.



Sean Dague <sean at dague.net> wrote:

>So we've current got a problem in Tempest that's due to the 
>incompatibility of 2 test tools that we want to use (flake8 & testr). 
>This can best be exemplified in the following review - 
>https://review.openstack.org/#/c/29293/ - which tries to fix the flake8 
>issue, but will cause an issue in switching to testr.
>
>
>Crux of the problem - Tempest needs to have a config file, and we need 
>it parsed early
>========
>
>Unlike unit tests, Tempest is designed as a tool that you run against a 
>living cloud. In the devstack-gate that's a cloud of our own creation, 
>but in the general case tempest should be runnable against any cloud. 
>That means you have to configure it to match your OpenStack cloud 
>configuration, including credentials, images, flavors, and services that 
>you may or may not have in your environment.
>
>This is done in a tempest.conf file.
>
>Under nose we had a setup_package hook, which let us do the config file 
>parsing early, and once for the entire run. As we made tempest testr 
>compatible, we had to stop using that hook, as it doesn't exist for 
>testr. Instead this all went into __init__.py files.
>
>This causes issues with flake8, because flake8 evals python code as part 
>of it's scan (largely to check the import rules IIRC). So for anyone 
>that's seen a tempest pep8 run since the flake8 transition will notice 
>the following 
>(http://logs.openstack.org/29395/4/check/gate-tempest-pep8/3408/console.html)
>
>Config file /etc/tempest/tempest.conf not found
>2013-05-21 17:03:43.195 | Traceback (most recent call last):
>2013-05-21 17:03:43.196 |   File 
>"/home/jenkins/workspace/gate-tempest-pep8/.tox/pep8/local/lib/python2.7/site-packages/hacking/core.py", 
>line 184, in is_module_for_sure
>2013-05-21 17:03:43.196 |     __import__(mod)
>2013-05-21 17:03:43.196 |   File 
>"/home/jenkins/workspace/gate-tempest-pep8/.tox/pep8/local/lib/python2.7/site-packages/tempest/api/compute/base.py", 
>line 31, in <module>
>2013-05-21 17:03:43.196 |     class 
>BaseComputeTest(tempest.test.BaseTestCase):
>2013-05-21 17:03:43.196 |   File 
>"/home/jenkins/workspace/gate-tempest-pep8/.tox/pep8/local/lib/python2.7/site-packages/tempest/api/compute/base.py", 
>line 34, in BaseComputeTest
>2013-05-21 17:03:43.197 |     conclusion = compute.generic_setup_package()
>2013-05-21 17:03:43.197 |   File 
>"/home/jenkins/workspace/gate-tempest-pep8/.tox/pep8/local/lib/python2.7/site-packages/tempest/api/compute/__init__.py", 
>line 42, in generic_setup_package
>2013-05-21 17:03:43.197 |     os = clients.Manager()
>2013-05-21 17:03:43.197 |   File 
>"/home/jenkins/workspace/gate-tempest-pep8/.tox/pep8/local/lib/python2.7/site-packages/tempest/clients.py", 
>line 279, in __init__
>2013-05-21 17:03:43.197 |     self.token_client = 
>TOKEN_CLIENT[interface](self.config)
>2013-05-21 17:03:43.198 |   File 
>"/home/jenkins/workspace/gate-tempest-pep8/.tox/pep8/local/lib/python2.7/site-packages/tempest/services/identity/json/identity_client.py", 
>line 225, in __init__
>2013-05-21 17:03:43.198 |     if 'tokens' not in auth_url:
>2013-05-21 17:03:43.199 | TypeError: argument of type 'NoneType' is not 
>iterable
>
>
>We had to disable exiting if we couldn't find the config file to get 
>flake8 in, which was the previous behavior.
>
>
>Why do we want tempest.conf parsed early?
>========
>2 reasons.
>
>1 - it's really a read only data parse, and moving it to class 
>initialization ends up meaning we reparse the same config file many many 
>times. Which is inefficient. It's still in the CI noise level though, so 
>if we had to be inefficient, so be it.
>
>2 - That's because we'd like the config early to be able to use config 
>variables in skipIf conditions:
>
>i.e. 
>https://github.com/openstack/tempest/blob/master/tempest/api/compute/servers/test_server_actions.py#L150
>
>     @attr(type='smoke')
>     @testtools.skipIf(not resize_available, 'Resize not available.')
>     def test_resize_server_confirm(self):
>         # The server's RAM and disk space should be modified to that of
>         # the provided flavor
>
>         previous_flavor_ref, new_flavor_ref = \
>             self._detect_server_image_flavor(self.server_id)
>
>         resp, server = self.client.resize(self.server_id, new_flavor_ref)
>         self.assertEqual(202, resp.status)
>         self.client.wait_for_server_status(self.server_id, 'VERIFY_RESIZE')
>
>         self.client.confirm_resize(self.server_id)
>         self.client.wait_for_server_status(self.server_id, 'ACTIVE')
>
>         resp, server = self.client.get_server(self.server_id)
>         self.assertEqual(new_flavor_ref, int(server['flavor']['id']))
>
>Which would be a bigger rewrite to work around a testr limitation.
>
>So right now we're kind of caught between a rock and a hard place, as in 
>the current state we can't satisfy both flake8 and testr without another 
>chunk of rewrite. I'd really appreciate thoughts, especially from folks 
>that have been on various parts of this as to how we get to a point of 
>not stack tracking on flake8, and still being able to move forward on testr.
>
>	-Sean
>
>-- 
>Sean Dague
>http://dague.net
>
>_______________________________________________
>openstack-qa mailing list
>openstack-qa at lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>