[openstack-qa] narrowing tempest to specific error codes

Sean Dague sdague at linux.vnet.ibm.com
Fri Nov 16 12:49:32 UTC 2012

After talking with Dave yesterday I was looking through 
https://bugs.launchpad.net/nova/+bug/1079210 which has a link to a log 
file with a lot of stack traces in nova-api

I picked one at mostly random and worked through it, now fixed here - 
https://bugs.launchpad.net/nova/+bug/1079387, tempest change to confirm 
the behavior ready for review here - https://review.openstack.org/#/c/16300/

The root cause of Tempest passing, but nova exploding, is the fact that 
our negative tests in Tempest are really broad. Any fail is acceptable. 
So in this case a nova-api 500 and stack trace was deemed a-ok.

But most of these failures should happen in a controlled way, with a 
known error state. The value of tempest in the gate goes way up in these 

I think we need a blueprint for making error codes specific in tempest 
during the grizzly cycle. Thoughts?

We should also be a bit more careful on reviews to make sure people are 
being specific about their fail conditions in the future.


Sean Dague
IBM Linux Technology Center
email: sdague at linux.vnet.ibm.com
alt-email: sldague at us.ibm.com

More information about the openstack-qa mailing list