Open Stack

Sean Dague <sean at dague.net> wrote:

> I've been looking at the following patch series -
> https://review.openstack.org/#/c/131691/13 for removing database
> requirements from some tests.
> 
> I whole heartedly support getting DB usage out of tests, but I'd like to
> make sure that we don't create new challenges in the process. The
> conditional create_db parameter in test functions adds a bit more
> internal test complexity than I think we should have.
> 
> I'd like to propose instead DB usage should be removed per test Class as
> an atomic unit. If that turns into too large a patch that probably means
> breaking apart the test class into smaller test classes first.
> 
> The other thing to be careful in understanding the results of timing
> tests. The way the database fixture works it caches the migration
> process -
> https://github.com/openstack/nova/blob/master/nova/tests/fixtures.py#L206
> 
> That actually means that the overhead of the db-migration sync is paid
> only once per testr worker (it's 1s on my fast workstation, might be 2s
> on gate nodes). The subsequence db setups for tests 2 -> N in the worker
> only take about 0.020s on my workstation (scale appropriately). So
> removing all the unneeded db setup code is probably only going to save
> ~30s over an entire test run.
> 
> Which doesn't mean it shouldn't be done, there are other safety reasons
> we shouldn't let every test randomly punch data into the db and still
> pass. But time savings should not be the primary motivator here, because
> it's actually not nearly as much gain as it looks like from running only
> a small number of tests.

I have a stalled patch for oslo.db that would provide for a new db fixture such that tests wouldn’t have to worry too much about when fixtures are set up or when per-test data is loaded; the system uses testresources to organize setup of schemas and teardown of data within longer-lived schemas in the most efficient way possible, across any number of backends:  https://review.openstack.org/#/c/120870/.   Simplifying and bringing great consistency to the fixtures of participating projects and reducing the reliance upon SQLite in tests are the goals of this patch.  The patch is stalled not for any particular reason, except perhaps some lingering discomfort with the OptimisingTestSuite approach that necessitates patching into unittest internals in order to reorder whole groups of tests.

So yes, DB-specific tests and tests without DB absolutely have to remain broken up along class lines; the migration of an individual DB-test to a non-DB test should involve moving it to a different class that isn’t a DB-related fixture.  







> 
> 	-Sean
> 
> -- 
> Sean Dague
> http://dague.net
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev