Open Stack

On Tue, Jan 22, 2013 at 01:08:15PM +1300, Robert Collins wrote:
> In addition to Monty's reply...
> 
> On 22 January 2013 02:42, Daniel P. Berrange <berrange at redhat.com> wrote:
> > In Folsom if you ran Nova's unit tests you'd get an informative display of
> > what was being run
> >
> > $ ./run_tests.sh test_libvirt_vif
> >
> 
> > Slowest 3 tests took 1.08 secs:
> >     0.98    LibvirtVifTestCase.test_bridge_driver
> >     0.07    LibvirtVifTestCase.test_ovs_ethernet_driver
> >     0.03    LibvirtVifTestCase.test_model_xen
> > ----------------------------------------------------------------------
> > Ran 9 tests in 1.227s
> 
> You can get this (until run_tests is fixed) by running
> tools/with_venv.sh testr slowest
> 
> and it will print out the slowest tests from the DB. I suspect you
> want slowest-from-the-last-run  - please file (or let me know you'd
> like me to file one) a bug on testrepository and I can add that, its
> an overlooked use case but easily satisfied.

Actually, I'm not too worried about the 'slowest tests' side of
thing. As mentioned in my other reply in this thread, I'm more
just interested in an indication that it is "doing something",
whatever that may be.

> As for verbose output. Fixing
> https://bugs.launchpad.net/testtools/+bug/872906 would make it very
> easy for testr to change its verbosity, for folk that want chatty
> output. (There are other alternatives such as tribunal or running
> tests with --subunit | subunit2pyunit, but I'm here to support folk
> using my tools, not tell them how to do it ;) [testrepository is a
> personal project of mine, built out of frustration with the need to
> continually rewrite the same darn code in different test
> environments])
> 
> In terms of detailed changes, testr already grabs everything needed to
> deliver what you want, but its UI is tuned to the workflow established
> by devs on big projects - 10's of thousands of test cases, highly
> parallel test execution - and some of the opinionated choices need to
> become opinionated defaults instead. I totally support that, and would
> be delighted to provide advice on where/how to change things to make
> different configurations easy to use ['--subunit | subunit2pyunit' is
> not 'easy to use'] without compromising the reliability of the CI
> environment.

I appreciate that testr is aiming to be excellant at handling the case
of big projects with 10,000's or test cases. Even in such projects
though, individual developers will spend the majority of their  time
just running a handful of test cases. eg I mostly just run the
test_libvirt* and test_virt*. Only right before submitting, do I
run the complete test suite. So I just want to make sure we're taking
account of both needs

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|