Open Stack

On 10/07/2012 10:51 AM, Salvatore Orlando wrote:
>
> The unit testd were transferred to file because in_memory db was 
> leaking memory, causing the jenkins job to go OOM. So this is ptobably 
> not going to help a lot as the infra team informed us unit tests 
> should be able to run on a 2gb ram machine.
>
> The flipside of this id that now they take longer, and sometimes the 
> jenkins job kills itself because cannot finish in 30 minutes.
> So, as you native english speakers say, we're between a rock and a 
> hard place.
>

True. I agree with you.
>
> Assuming we can find a way to stem the memory leak, I would like to 
> kerp in memory unit tests. I tried, but to no avail. It seems sqlite 
> does not free memory even if you explicitly destroy tables.
>

I will try and look around to see if there is some way of freeing the 
memory. Do you know which sqlalchemy version is used.  
http://www.sqlalchemy.org/blog/ indicates that there were some memory 
leak issues fixed.
>
> Otherwise, Bob proposed to mount the sqlite db file into a ram fs. I 
> have not spent any time on thid, but assumung it is feasible for the 
> infra team, I think this would be the way to go.
>

I tried this - from what I read it indicated that one should use the 
cache pragma. This reduces the time to half but it is still very high - 
for example 2 seconds for a unit test.
>
> Last resort would be to shorten the number of tests by removing 
> test_<pluginname>_plugin modules and appropriate modules which run 
> only unit tests for covering methods overriden by the specific plugin.
>
> Il giorno 07/ott/2012 09:09, "Gary Kotton" <gkotton at redhat.com 
> <mailto:gkotton at redhat.com>> ha scritto:
>
>     Hi,
>     There have been a number of issues regarding unit tests as of late
>     - the main problem being that they are taking too long. The reason
>     for the increased test time was using a file instead of memory. I
>     have tried a number of things to improve the test times but to no
>     avail (for example defining the pragma cache which I understand
>     uses mmap for the file - this reduced the time of an average test
>     from 4 seconds to 2 seconds. When using memory this was 0.10
>     secs). I have reverted them back to the in memory and suggest that
>     we discuss on how to tackle the problem in the future. I think
>     that it is better that we have the tests running and try and work
>     on a better long term solution off line. I guess that we can
>     discuss this at summit. Please look at -
>     https://review.openstack.org/14135
>     Thanks
>     Gary
>
>     _______________________________________________
>     OpenStack-dev mailing list
>     OpenStack-dev at lists.openstack.org
>     <mailto:OpenStack-dev at lists.openstack.org>
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20121007/938176f1/attachment.html>