Open Stack

On 12/10/2012 10:57 AM, Vishvananda Ishaya wrote:
> 
> On Dec 10, 2012, at 10:04 AM, Sean Dague <sdague at linux.vnet.ibm.com>
> wrote:
> 
>> On 12/10/2012 11:58 AM, Adam Young wrote:
>>> On 12/10/2012 11:36 AM, Kevin L. Mitchell wrote:
>>>> On Mon, 2012-12-10 at 09:59 +1000, Matthew Sherborne wrote:
>>>>> Why don't we create and destroy a test sqlite in memory
>>>>> database for the tests instead of stubbing out the DB calls
>>>>> ?
>>>> Stubbing out the DB calls makes perfect sense for the unit
>>>> tests, which honestly should be (but isn't) the bulk of our
>>>> test suite.  Using an in-memory database is exactly what should
>>>> be happening for the function tests, and Monty's comment
>>>> indicates that it is…
>>>> 
>>> A quibble:  We do the sqlite in memory thing for most unit tests
>>> in Keystone.  I see it as our Mock DB implementation, and it
>>> provides a better unit test than reimplementing a halfway set of
>>> sql mock objects.
>> 
>> Given the complexity of the nova db, and the number of migrations
>> that build up (50+ in Essex -> Grizzly) there are performance
>> implications to keep doing that over and over again for all the
>> unit tests, especially the ones that are separated enough from the
>> database that their code should be tested in isolation.
> 
> It may be a small performance hit, but we currently have a fresh copy
> of the database for every single test run, so that ship has sailed.
> FYI the migrations run once and the db is just reset to the initial
> state between each test.

That is correct, we do migrations once and then save the results to use
as the initial state for each test.

Also, we're REALLY REALLY close to having parallel test running finished
for nova, which is currently showing a 5x performance improvement over
the current test run on my laptop, so the performance hit of doing that
properly (which is required for parallelization) is more than made up
for by parallel operation.