Open Stack

Something that's currently just far to manual to figure out is how testr
is functioning in the parallel case, which tests are running on which
process, and when does one of the pipelines run out of tests. The testr
assumption that you don't want output unless there is a failure is a
really bad assumption, because it's important to know that things are
progressing, especially in complicated systems.

In tempest we actually have a wrapper that gives us a stream. But it's
got a couple of fundamental flaws.

First - it is very clear that the expected behavior of start / end of
tests doesn't work:

2014-01-24 00:49:59.005 |
tempest.api.image.v1.test_images.CreateRegisterImagesTest.test_register_image_with_min_ram[gate]
2014-01-24 00:49:59.005 |
tempest.api.image.v1.test_images.CreateRegisterImagesTest.test_register_image_with_min_ram[gate]
... ok
2014-01-24 00:49:59.121 |
tempest.api.image.v1.test_images.CreateRegisterImagesTest.test_register_remote_image[gate]
2014-01-24 00:49:59.122 |
tempest.api.image.v1.test_images.CreateRegisterImagesTest.test_register_remote_image[gate]
... ok
2014-01-24 00:49:59.932 |
tempest.api.image.v1.test_images.CreateRegisterImagesTest.test_register_then_upload[gate]
2014-01-24 00:49:59.933 |
tempest.api.image.v1.test_images.CreateRegisterImagesTest.test_register_then_upload[gate]
... ok

These tests take > 1msec, all that's happening is the start message is
being printed with the end message, which is kind of useless.

Question #1: how do we either make start message be when things actually
start? or turn it off entirely?


Second - it's not entirely obvious how unbalanced the pipelines are, and
that would be extremely useful to see in the output. A starting point
for a UI would be to add...

" " x (5 * pipelinenumber)

To the test results. Then you'd have a visual waterfall to the front of
the tests, and could clearly see pipeline differences.

Question #2: where would this be injectable in the stream of testtools /
subunit / testr / unittest?

	-Sean

-- 
Sean Dague
Samsung Research America
sean at dague.net / sean.dague at samsung.com
http://dague.net

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140125/dcd9075a/attachment.pgp>