[openstack-dev] [Quantum] Unit tests and memory consumption

Gary Kotton gkotton at redhat.com
Sun Nov 4 14:26:57 UTC 2012


On 10/24/2012 03:24 PM, Gary Kotton wrote:
> On 10/24/2012 12:20 AM, Monty Taylor wrote:
>> I believe that we can address this for both run_tests and tox with some
>> of the testtools/fixtures stuff clark has been playing with. We'll poke
>> at it tomorrow (he's out for the day) and see if we can get an approach
>> that would make everyone happy.
>
I tried playing around with the flags in the tox.ini file. There are 
options to run this in multiple processes. There are two flags that are 
of interest:

1. NOSE_PROCESS_RESTARTWORKER: - This is documented as follows:
"--process-restartworker
                         If set, will restart each worker process once their
                         tests are done, this helps control memory leaks 
from
                         killing the system. [NOSE_PROCESS_RESTARTWORKER]"
2. NOSE_PROCESSES: - This is documented as follows:
"--processes=NUM       Spread test run among this many processes. Set a
                         number equal to the number of processors or 
cores in
                         your machine for best results. [NOSE_PROCESSES]"

The problem that I have is that when the first variable is set nothing 
is done, this is really dependent on the second. The problem when 
setting the second is that I get the following exception:

garyk at linus:~/quantum$ tox
GLOB sdist-make: /home/garyk/quantum/setup.py
py26 create: /home/garyk/quantum/.tox/py26
ERROR: InterpreterNotFound: python2.6
py27 sdist-reinst: /home/garyk/quantum/.tox/dist/quantum-2013.1.zip
py27 runtests: commands[0]

/home/garyk/quantum/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/ext/declarative.py:1343: 
SAWarning: The classname 'NetworkBinding' is already in the registry of 
this declarative base, mapped to <class 
'quantum.plugins.openvswitch.ovs_models_v2.NetworkBinding'>
   _as_declarative(cls, classname, cls.__dict__)

Traceback (most recent call last):
   File ".tox/py27/bin/nosetests", line 9, in <module>
     load_entry_point('nose==1.2.1', 'console_scripts', 'nosetests')()
   File 
"/home/garyk/quantum/.tox/py27/local/lib/python2.7/site-packages/nose/core.py", 
line 118, in __init__
     **extra_args)
   File "/usr/lib/python2.7/unittest/main.py", line 95, in __init__
     self.runTests()
   File 
"/home/garyk/quantum/.tox/py27/local/lib/python2.7/site-packages/nose/core.py", 
line 197, in runTests
     result = self.testRunner.run(self.test)
   File 
"/home/garyk/quantum/.tox/py27/local/lib/python2.7/site-packages/nose/plugins/multiprocess.py", 
line 357, in run
     testQueue = Queue()
   File "/usr/lib/python2.7/multiprocessing/managers.py", line 667, in temp
     token, exp = self._create(typeid, *args, **kwds)
   File "/usr/lib/python2.7/multiprocessing/managers.py", line 565, in 
_create
     conn = self._Client(self._address, authkey=self._authkey)
   File "/usr/lib/python2.7/multiprocessing/connection.py", line 175, in 
Client
     answer_challenge(c, authkey)
   File "/usr/lib/python2.7/multiprocessing/connection.py", line 413, in 
answer_challenge
     message = connection.recv_bytes(256)         # reject large message
IOError: [Errno 11] Resource temporarily unavailable

Has anyone ever encountered something like this.
Thanks
Gary






More information about the OpenStack-dev mailing list