After talking with Dave yesterday I was looking through https://bugs.launchpad.net/nova/+bug/1079210 which has a link to a log file with a lot of stack traces in nova-api I picked one at mostly random and worked through it, now fixed here - https://bugs.launchpad.net/nova/+bug/1079387, tempest change to confirm the behavior ready for review here - https://review.openstack.org/#/c/16300/ The root cause of Tempest passing, but nova exploding, is the fact that our negative tests in Tempest are really broad. Any fail is acceptable. So in this case a nova-api 500 and stack trace was deemed a-ok. But most of these failures should happen in a controlled way, with a known error state. The value of tempest in the gate goes way up in these cases. I think we need a blueprint for making error codes specific in tempest during the grizzly cycle. Thoughts? We should also be a bit more careful on reviews to make sure people are being specific about their fail conditions in the future. -Sean -- Sean Dague IBM Linux Technology Center email: sdague at linux.vnet.ibm.com alt-email: sldague at us.ibm.com