[openstack-dev] [Fuel] Test runner for python tests and parallel execution
chdent at redhat.com
Sun Nov 9 14:43:30 UTC 2014
On Sat, 8 Nov 2014, Robert Collins wrote:
> What changes do you want to see in the ui?
I don't want to hijack the thread too much so I hope Dmitriy will join
back in but for me there are two aspects of the existing experience
that don't work out well. I suspect many of these situations can be
resolved with more info (that is, the bug is in my ignorance, not in
* Lack of transparency on how to manage verbosity and output handling
during a test run. Obviously the output during an unobserved run is
going to need to be different from what I as a devloper want while
doing an observed run.
In the latter case I want to know, while it is happening, which
tests have been discovered, which one is happening right now, and a
sense of the status of the current assert.
I want, at my option, to spew stderr and stdout directly without
interference so I can do unhygenic debugging.
Essentially, I want to be able to discover the flags, arguments and
toosl that allow me to use tests as an ad hoc development aid, not
I know this is possible with the existing tools, it's just not easy
nor easy (for me) to discover.
* The current testing code and tools are, to use that lovely saw, "hard
to reason about". Which for me equates to "hard to read".
This is perhaps because I'm not particular wed to _unit_ tests,
_unittest_ formed tests, or concepts such as test isolation and
hates mocks. I see the value of these things but I think it is
easy for them to be overused and make other purposes more difficult.
I threw this up on the list a little while, which is related:
Same complaints and a hope that we won't have the same over-emphasis
when we move to in tree testing.
Summary: I think we need to spend some time and thought on improving
the usefulness of tests, testing and testing tools for someone working
on a feature or a bug _right now_. That is: tests run by humans should
be as frictionless as possible so that bugs are caught and fixed
before test suites are ever run by robots.
 And with luck will help create more effective and usable code.
 Yes, I believe in TDD.
Chris Dent tw:@anticdent freenode:cdent
More information about the OpenStack-dev