Open Stack

On Sat, 8 Nov 2014, Robert Collins wrote:

> What changes do you want to see in the ui?

I don't want to hijack the thread too much so I hope Dmitriy will join
back in but for me there are two aspects of the existing experience
that don't work out well. I suspect many of these situations can be
resolved with more info (that is, the bug is in my ignorance, not in
the software).

* Lack of transparency on how to manage verbosity and output handling
   during a test run. Obviously the output during an unobserved run is
   going to need to be different from what I as a devloper want while
   doing an observed run.

   In the latter case I want to know, while it is happening, which
   tests have been discovered, which one is happening right now, and a
   sense of the status of the current assert.

   I want, at my option, to spew stderr and stdout directly without
   interference so I can do unhygenic debugging.

   Essentially, I want to be able to discover the flags, arguments and
   toosl that allow me to use tests as an ad hoc development aid, not
   post hoc.

   I know this is possible with the existing tools, it's just not easy
   nor easy (for me) to discover.

* The current testing code and tools are, to use that lovely saw, "hard
   to reason about". Which for me equates to "hard to read".

   This is perhaps because I'm not particular wed to _unit_ tests,
   _unittest_ formed tests, or concepts such as test isolation and
   hates mocks. I see the value of these things but I think it is
   easy for them to be overused and make other purposes more difficult.

   I threw this[1] up on the list a little while, which is related:
   Same complaints and a hope that we won't have the same over-emphasis
   when we move to in tree testing.

Summary: I think we need to spend some time and thought on improving
the usefulness of tests, testing and testing tools for someone working
on a feature or a bug _right now_. That is: tests run by humans should
be as frictionless as possible so that bugs are caught[2] and fixed
before test suites are ever run by robots.

[1] https://tank.peermore.com/tanks/cdent-rhat/SummitFunctionalTesting

[2] And with luck will help create more effective and usable code[3].

[3] Yes, I believe in TDD.
-- 
Chris Dent tw:@anticdent freenode:cdent
https://tank.peermore.com/tanks/cdent