Open Stack

@Alex: testr is exactly what I had in mind. thanks for pointer to
eatmydata, I've just tried it, and test run time is down about 40%.
Awesome!

@Mark: The full request stack, which we use in several places, came
handy and easy for Essex and Folsom. It is far too easy now to forget
to remove some garbage and cause massive leaks (on this note... you
should have added log handlers to that list - they eat up a lot of
memory!). It makes sense to execute the same set of tests against each
plugin, but the benefit is very little when compared to the cost, as
we are now running 10 times (once for each plugin), code which is
pretty much the same. We are suffering to a common problem which is
the result of a mistake I often make, ie: confusing unit testing with
functional testing.

The refactoring will necessarily involve plugin maintainers, as unit
tests now will be dictated by the logic specific to each plugin. So it
will definitely need to slip into Havana.

Salvatore


On 6 February 2013 21:25, Mark McClain <mark.mcclain at dreamhost.com> wrote:
> I'd like to see us not do a full request stack with testing the plugins.  A lot of our leaks are a combination of leftover http routing, validation, and database setups.  This is on my list of blueprints to change in Havana (it would be too disruptive to massively refactor now).
>
> mark
>
> On Feb 6, 2013, at 3:21 PM, Salvatore Orlando <sorlando at nicira.com> wrote:
>
>> Hi,
>>
>> As some of you have noticed, the number of unit tests arose to above
>> 3,000 with recent merges.
>> The XML support is adding most of them, by doing something similar to
>> what the API tests for Quantum v1 did (execute the whole test cases
>> with both format).
>> While I understand and support this approach, there are two things to be noted:
>> - Unit test execution times have more than doubled
>> - With Quantum v2 we adopted a slightly different approach, by
>> separating API and DB tests. The formers tests (that we have for the
>> core API as well as many extensions) are the ones aimed at validating
>> correct serialization/deserialization. If we keep following this
>> approach, we might be able to save a lot of format-specific unit tests
>> for DB logic. From my perspective I don't think we will be losing any
>> coverage - but I might be missing something fundamental.
>>
>> I recall Aaron and BobK in recent posts were suggesting to move to
>> parallel testing; this one of the first things we should address in
>> Havana. In the meanwhile, let me know if you think we can do something
>> to reduce the amount of resources used by unit tests.
>>
>> I am also noticing that the memory leaks which were reduced a lot
>> during G-1 are hitting again, on low memory workers (2Gb); probably no
>> new leak has been introduced, it's just that the effect of the few
>> ones which were left it's now being felt more.
>>
>> Salvatore
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev