Open Stack

On 03/10/2014 02:31 PM, Zane Bitter wrote:
> Fewer logs is hardly ever what you want when debugging a unit test.
> 
> I think what John is looking for is a report at the end of each test run 
> that just lists the tests that failed instead of all the details (like 
> `testr failing --list`), or perhaps the complete list of tests with the 
> coloured pass/fail like IIRC nose does. Since testr's killer feature is 
> to helpfully store all of the results for later, maybe this output is 
> all you need in the first instance (along with a message telling the 
> user what command to run to see the full output, of course), or at least 
> that could be an option.

> It sounds like what John wants to do is pass a filter to something like 
> `testr failing` or `testr last` to only report a subset of the results, 
> in much the same way as it's possible to pass a filter to `testr` to 
> only run a subset of the tests.

Common vocabulary is essential to discuss this. A test result as emitted
by subunit is a metadata collection indexed by keys. To get the list of
failing tests one iterates over the set of results and looks for the
absense of the "successful" key in the result. That's the set of test
failures, as such it's a filter on the test results.

Therefore I see filtering as the act of producing a subset of the test
results (i.e, only those failing, or only those whose names match a
regexp, or the intersection of those). That is a filtered result set.
The filtering is performed by examining the key/value in each result
metadata to yield a subset of the results.

Next you have to display that filtered result set. When each result is
displayed one should be able to specify which pieces of metadata get
displayed. In my mind that's not filtering, it's a display option. One
common display option would be to emit only the test name. Another might
be to display the test name and the captured log data. As it stands now
it seems to display every piece of metadata in the result which is what
is producing the excessive verbosity.

Hopefully I'm making sense, yes/no?

-- 
John