Open Stack

On 1/28/2015 10:59 AM, Chen CH Ji wrote:
> Sorry for late reply and thanks for bring this out, I agree the
> create_db flag will increase the complexity
> so I might do some PoC and write a spec to do it next release
>
> For this sentence, I don't fully understand, are you suggesting to every
> db usage remove should be a
> patch for a test class? thanks a lot
> /I'd like to propose instead DB usage should be removed per test Class as
> an atomic unit./
>
> Best Regards!
>
> Kevin (Chen) Ji 纪 晨
>
> Engineer, zVM Development, CSTL
> Notes: Chen CH Ji/China/IBM at IBMCN   Internet: jichenjc at cn.ibm.com
> Phone: +86-10-82454158
> Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian
> District, Beijing 100193, PRC
>
> Inactive hide details for Sean Dague ---01/24/2015 07:13:45 PM---I've
> been looking at the following patch series - https://reviSean Dague
> ---01/24/2015 07:13:45 PM---I've been looking at the following patch
> series - https://review.openstack.org/#/c/131691/13 for rem
>
> From: Sean Dague <sean at dague.net>
> To: "OpenStack Development Mailing List (not for usage questions)"
> <openstack-dev at lists.openstack.org>
> Cc: Chen CH Ji/China/IBM at IBMCN
> Date: 01/24/2015 07:13 PM
> Subject: [nova] proposal for unwinding database usage from tests
>
> ------------------------------------------------------------------------
>
>
>
> I've been looking at the following patch series -
> https://review.openstack.org/#/c/131691/13 for removing database
> requirements from some tests.
>
> I whole heartedly support getting DB usage out of tests, but I'd like to
> make sure that we don't create new challenges in the process. The
> conditional create_db parameter in test functions adds a bit more
> internal test complexity than I think we should have.
>
> I'd like to propose instead DB usage should be removed per test Class as
> an atomic unit. If that turns into too large a patch that probably means
> breaking apart the test class into smaller test classes first.
>
> The other thing to be careful in understanding the results of timing
> tests. The way the database fixture works it caches the migration
> process -
> https://github.com/openstack/nova/blob/master/nova/tests/fixtures.py#L206
>
> That actually means that the overhead of the db-migration sync is paid
> only once per testr worker (it's 1s on my fast workstation, might be 2s
> on gate nodes). The subsequence db setups for tests 2 -> N in the worker
> only take about 0.020s on my workstation (scale appropriately). So
> removing all the unneeded db setup code is probably only going to save
> ~30s over an entire test run.
>
> Which doesn't mean it shouldn't be done, there are other safety reasons
> we shouldn't let every test randomly punch data into the db and still
> pass. But time savings should not be the primary motivator here, because
> it's actually not nearly as much gain as it looks like from running only
> a small number of tests.
>
> -Sean
>
> --
> Sean Dague
> http://dague.net
>
> /(See attached file: signature.asc)/
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

I take that to mean doing something like this:

https://review.openstack.org/#/c/142655/

But I don't want to speak for Sean.  But that would be a lot of work for 
some test classes, so I'd think it'd have to be phased, which is 
happening in some places, e.g. danpb was doing some of that with the 
libvirt unit test refactoring, and we've done some of that with the 
compute manager tests.  test_compute_mgr is huge though so refactoring 
and moving tests to NoDB will take time (test_neutronv2 also has some of 
this similar issue, i.e. we want to get away from the DB and mega-mox in 
setUp and move to using mock, so there is a separate test class in that 
module which uses Mock and NoDB and new tests should live there unless 
we can just update small parts of old tests).

So this is kind of like the mox->mock conversion, it's OK to update 
existing test cases for bug fixes, but if you have new unit tests (new 
test cases/classes), do them in mock (and do them w/o the DB).

-- 

Thanks,

Matt Riedemann