Open Stack

I've been looking at the following patch series -
https://review.openstack.org/#/c/131691/13 for removing database
requirements from some tests.

I whole heartedly support getting DB usage out of tests, but I'd like to
make sure that we don't create new challenges in the process. The
conditional create_db parameter in test functions adds a bit more
internal test complexity than I think we should have.

I'd like to propose instead DB usage should be removed per test Class as
an atomic unit. If that turns into too large a patch that probably means
breaking apart the test class into smaller test classes first.

The other thing to be careful in understanding the results of timing
tests. The way the database fixture works it caches the migration
process -
https://github.com/openstack/nova/blob/master/nova/tests/fixtures.py#L206

That actually means that the overhead of the db-migration sync is paid
only once per testr worker (it's 1s on my fast workstation, might be 2s
on gate nodes). The subsequence db setups for tests 2 -> N in the worker
only take about 0.020s on my workstation (scale appropriately). So
removing all the unneeded db setup code is probably only going to save
~30s over an entire test run.

Which doesn't mean it shouldn't be done, there are other safety reasons
we shouldn't let every test randomly punch data into the db and still
pass. But time savings should not be the primary motivator here, because
it's actually not nearly as much gain as it looks like from running only
a small number of tests.

	-Sean

-- 
Sean Dague
http://dague.net

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 465 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150124/bbab34d4/attachment.pgp>