[openstack-dev] [ceilometer] [gnocchi] monitoring unexpected success of xfail designated tests

Chris Dent chdent at redhat.com
Fri Mar 6 16:31:00 UTC 2015

On Fri, 6 Mar 2015, Julien Danjou wrote:

> On Fri, Mar 06 2015, Chris Dent wrote:
>> Gnocchi has already started using pretty_tox.sh, so as long as there
>> is a recent version of subunit-trace (from tempest-lib) installed,
>> gnocchi test runs will tell when there has been an unexpected
>> success.
> Are you saying that unexpected success is not a failure?

testr will announce "commands succeeded" when there is a uxsuccess in the
result set. There me be some special sauce we can give it (or
perhaps subunit.run) to say otherwise, but as things are currently
written in the tox files, uxsuccess means pass.

> That sounds wrong. Where should we fix that?

It looks like the problem is in subunit (this is with a locally
modified gabbi test):

     $ for i in testtools subunit ; \
       do python -m $i.run discover gnocchi.tests.gabbi &>/dev/null || \
           echo "$i catches uxsuccess as fail" ; \
     testtools catches uxsuccess as fail

I've filed a bug: https://bugs.launchpad.net/subunit/+bug/1429196

Chris Dent tw:@anticdent freenode:cdent

More information about the OpenStack-dev mailing list