[openstack-dev] per process output from testr parallel?

Robert Collins robertc at robertcollins.net
Wed Jan 29 21:19:37 UTC 2014

On 26 January 2014 02:25, Sean Dague <sean at dague.net> wrote:
> Something that's currently just far to manual to figure out is how testr
> is functioning in the parallel case, which tests are running on which
> process, and when does one of the pipelines run out of tests. The testr
> assumption that you don't want output unless there is a failure is a
> really bad assumption, because it's important to know that things are
> progressing, especially in complicated systems.

IMO There are three distinct use cases - we need to be able to tell
what was concurrent (the saved stream should do that post-hoc).
Secondly we need to identify hangs and so on - human scale. Lastly we
need automation around identifying broken things...

> In tempest we actually have a wrapper that gives us a stream. But it's
> got a couple of fundamental flaws.
> First - it is very clear that the expected behavior of start / end of
> tests doesn't work:
> 2014-01-24 00:49:59.005 |
> tempest.api.image.v1.test_images.CreateRegisterImagesTest.test_register_image_with_min_ram[gate]
> 2014-01-24 00:49:59.005 |
> tempest.api.image.v1.test_images.CreateRegisterImagesTest.test_register_image_with_min_ram[gate]
> ... ok

subunit2pyunit just routes everything through a regular python
'unittest.TextTestResult' which doesn't know about timing data or
concurrency. So we need to replace that.

> Question #1: how do we either make start message be when things actually
> start? or turn it off entirely?
> Second - it's not entirely obvious how unbalanced the pipelines are, and
> that would be extremely useful to see in the output. A starting point
> for a UI would be to add...
> " " x (5 * pipelinenumber)
> To the test results. Then you'd have a visual waterfall to the front of
> the tests, and could clearly see pipeline differences.

would (worker-N) be sufficient? Or do you want *'s/x's specifically?

> Question #2: where would this be injectable in the stream of testtools /
> subunit / testr / unittest?

Long term I defintely want a richer UI in testr, but right now the
most appropriate thing to do is to prototype the UI you need locally.


Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Converged Cloud

More information about the OpenStack-dev mailing list