Open Stack

On 1/24/2015 12:13 PM, Sean Dague wrote:
> I've been looking at the following patch series -
> https://review.openstack.org/#/c/131691/13 for removing database
> requirements from some tests.
>
> I whole heartedly support getting DB usage out of tests, but I'd like to
> make sure that we don't create new challenges in the process. The
> conditional create_db parameter in test functions adds a bit more
> internal test complexity than I think we should have.
>
> I'd like to propose instead DB usage should be removed per test Class as
> an atomic unit. If that turns into too large a patch that probably means
> breaking apart the test class into smaller test classes first.
>
> The other thing to be careful in understanding the results of timing
> tests. The way the database fixture works it caches the migration
> process -
> https://github.com/openstack/nova/blob/master/nova/tests/fixtures.py#L206
>
> That actually means that the overhead of the db-migration sync is paid
> only once per testr worker (it's 1s on my fast workstation, might be 2s
> on gate nodes). The subsequence db setups for tests 2 -> N in the worker
> only take about 0.020s on my workstation (scale appropriately). So
> removing all the unneeded db setup code is probably only going to save
> ~30s over an entire test run.
>
> Which doesn't mean it shouldn't be done, there are other safety reasons
> we shouldn't let every test randomly punch data into the db and still
> pass. But time savings should not be the primary motivator here, because
> it's actually not nearly as much gain as it looks like from running only
> a small number of tests.
>
> 	-Sean
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

Big +1 to doing this on a class by class basis, however, that can get 
ugly fast [1].

I commented on the change for test_compute to just move those to 
test_compute_api/mgr as appropriate, we should be freezing test_compute 
for the most part unless you're just modifying an existing test for some 
small wrinkle in the code.

I don't think we can completely change test_compute over to no-db in one 
change, that would be huge, we should just work on moving pieces over to 
those other modules that use NoDBTestCase (or move the bigger 
functional-type tests into nova.tests.functional).

One candidate is moving the big shelve/unshelve tests into 
test_compute_mgr/api, those have been a latent source of race bugs in 
the gate because of weird DB interactions/expectations/assumptions being 
borked by other tests running at the same time hitting the DB (because 
of so many different test classes that extend test_compute.BaseTestCase).

[1] https://review.openstack.org/#/c/142655/

-- 

Thanks,

Matt Riedemann