[openstack-dev] [Tempest] - how to debug

Sean Dague sean at dague.net
Fri Jun 14 11:27:54 UTC 2013


Start with that link, follow the fails from the Jenkins results post

http://logs.openstack.org/32178/2/check/gate-swift-pep8/2650 : SUCCESS 
in 42s
http://docs-draft.openstack.org/32178/2/check/gate-swift-docs/2480/doc/build/html/ 
: SUCCESS in 1m 09s
http://logs.openstack.org/32178/2/check/gate-swift-python26/2728 : 
SUCCESS in 2m 50s
http://logs.openstack.org/32178/2/check/gate-swift-python27/2726 : 
SUCCESS in 1m 08s
http://logs.openstack.org/32178/2/check/gate-tempest-devstack-vm-full/21501 
: FAILURE in 51m 26s
http://logs.openstack.org/32178/2/check/gate-tempest-devstack-vm-quantum/28634 
: FAILURE in 20m 13s
http://logs.openstack.org/32178/2/check/gate-tempest-devstack-vm-postgres-full/21208 
: FAILURE in 47m 29s
http://logs.openstack.org/32178/2/check/gate-grenade-devstack-vm/8777 : 
SUCCESS in 13m 53s (non-voting)


When you go to 
http://logs.openstack.org/32178/2/check/gate-tempest-devstack-vm-full/21501

You'll see a top level directory with console.html (the test run), and 
logs directory, which includes all the screen logs for all the services 
run by devstack.

Start with the console log to figure out where the fail happened....

The console log is big, so typically jump all the way to the end and 
work backwards. This will also ensure you find the actual critical fail, 
and not trip over some early red herrings that are things working as 
designed but with scary messages.

Working backwards on this run gets you the following:

2013-06-07 19:16:35.349 | 
======================================================================
2013-06-07 19:16:35.349 | FAIL: 
tempest.api.object_storage.test_object_services.ObjectTest.test_get_object_using_temp_url[gate]
2013-06-07 19:16:35.350 | 
----------------------------------------------------------------------
2013-06-07 19:16:35.381 | _StringException: Traceback (most recent call 
last):
2013-06-07 19:16:35.381 |   File 
"/opt/stack/new/tempest/tempest/api/object_storage/test_object_services.py", 
line 334, in test_get_object_using_temp_url
2013-06-07 19:16:35.382 |     metadata=metadata)
2013-06-07 19:16:35.382 |   File 
"/opt/stack/new/tempest/tempest/services/object_storage/account_client.py", 
line 62, in delete_account_metadata
2013-06-07 19:16:35.382 |     resp, body = self.post('', 
headers=headers, body=None)
2013-06-07 19:16:35.382 |   File 
"/opt/stack/new/tempest/tempest/common/rest_client.py", line 266, in post
2013-06-07 19:16:35.382 |     return self.request('POST', url, headers, 
body)
2013-06-07 19:16:35.382 |   File 
"/opt/stack/new/tempest/tempest/common/rest_client.py", line 394, in request
2013-06-07 19:16:35.382 |     resp, resp_body)
2013-06-07 19:16:35.382 |   File 
"/opt/stack/new/tempest/tempest/common/rest_client.py", line 444, in 
_error_checker
2013-06-07 19:16:35.382 |     raise exceptions.BadRequest(resp_body)
2013-06-07 19:16:35.382 | BadRequest: Bad request
2013-06-07 19:16:35.382 | Details: Header Line Too Long


So yes, this was related to your change, which changes max header 
lengths. At this point it's probably worth setting up tempest locally 
and poking this on your code to figure out if a tempest change is also 
needed to match the new behavior, or if this is actually legitimately a 
break in behavior.

That being said, I notice the review is -2ed by a person already as 
well, so getting that resolved would be important.

	-Sean

On 06/14/2013 02:38 AM, Gareth wrote:
> https://review.openstack.org/#/c/32178/
>
> here
>
>
> On Fri, Jun 14, 2013 at 2:25 PM, Eugene Nikanorov
> <enikanorov at mirantis.com <mailto:enikanorov at mirantis.com>> wrote:
>
>     Could you give a link to your review?
>
>     Thanks,
>     Euigne.
>
>
>     On Fri, Jun 14, 2013 at 10:04 AM, Gareth <academicgareth at gmail.com
>     <mailto:academicgareth at gmail.com>> wrote:
>
>         I'm not totally sure for that, and I have recheck many times,
>         resulting in same failure.
>
>         In that link, there's an internal error in glance upload, so I
>         want to find more reasons in it, but I don't want to review all
>         of tempest codes.
>
>
>         On Fri, Jun 14, 2013 at 1:07 PM, Eugene Nikanorov
>         <enikanorov at mirantis.com <mailto:enikanorov at mirantis.com>> wrote:
>
>             Hi,
>
>             If you're sure your code could not cause this, put 'recheck
>             no bug' comment to make Jenkins recheck your patch or
>             'reverify' to make it run the gate tests again.
>
>             Thanks,
>             Eugene.
>
>
>             On Fri, Jun 14, 2013 at 8:26 AM, Gareth
>             <academicgareth at gmail.com <mailto:academicgareth at gmail.com>>
>             wrote:
>
>                 Hi all
>
>                 I'm facing some issues in rechecking my patch:
>
>                 in this link,
>                 https://jenkins.openstack.org/job/gate-tempest-devstack-vm-postgres-full/22918/console,
>
>                 It seems trying to upload an image by Glance, but failed
>                 again and again, in all of vm. My my codes are just
>                 focusing on default settings in config file and some
>                 unit tests.
>
>                 I don't know how to work further now.
>
>                 Does anyone has same issues like mine?
>
>
>                 --
>                 Gareth
>
>                 /Cloud Computing, OpenStack, Fitness, Basketball/
>                 /OpenStack contributor/
>                 /Company: UnitedStack <http://www.ustack.com>/
>                 /My promise: if you find any spelling or grammar
>                 mistakes in my email from Mar 1 2013, notify me /
>                 /and I'll donate $1 or ¥1 to an open organization you
>                 specify./
>
>                 _______________________________________________
>                 OpenStack-dev mailing list
>                 OpenStack-dev at lists.openstack.org
>                 <mailto:OpenStack-dev at lists.openstack.org>
>                 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>             _______________________________________________
>             OpenStack-dev mailing list
>             OpenStack-dev at lists.openstack.org
>             <mailto:OpenStack-dev at lists.openstack.org>
>             http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
>         --
>         Gareth
>
>         /Cloud Computing, OpenStack, Fitness, Basketball/
>         /OpenStack contributor/
>         /Company: UnitedStack <http://www.ustack.com>/
>         /My promise: if you find any spelling or grammar mistakes in my
>         email from Mar 1 2013, notify me /
>         /and I'll donate $1 or ¥1 to an open organization you specify./
>
>         _______________________________________________
>         OpenStack-dev mailing list
>         OpenStack-dev at lists.openstack.org
>         <mailto:OpenStack-dev at lists.openstack.org>
>         http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>     _______________________________________________
>     OpenStack-dev mailing list
>     OpenStack-dev at lists.openstack.org
>     <mailto:OpenStack-dev at lists.openstack.org>
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> --
> Gareth
>
> /Cloud Computing, OpenStack, Fitness, Basketball/
> /OpenStack contributor/
> /Company: UnitedStack <http://www.ustack.com>/
> /My promise: if you find any spelling or grammar mistakes in my email
> from Mar 1 2013, notify me /
> /and I'll donate $1 or ¥1 to an open organization you specify./
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>


-- 
Sean Dague
http://dague.net



More information about the OpenStack-dev mailing list