Open Stack

On 11/05/2012 07:56 AM, Gary Kotton wrote:
> On 11/04/2012 10:48 PM, Monty Taylor wrote:
>>
>> On 11/04/2012 03:26 PM, Gary Kotton wrote:
>>> On 10/24/2012 03:24 PM, Gary Kotton wrote:
>>>> On 10/24/2012 12:20 AM, Monty Taylor wrote:
>>>>> I believe that we can address this for both run_tests and tox with
>>>>> some
>>>>> of the testtools/fixtures stuff clark has been playing with. We'll
>>>>> poke
>>>>> at it tomorrow (he's out for the day) and see if we can get an
>>>>> approach
>>>>> that would make everyone happy.
>>> I tried playing around with the flags in the tox.ini file. There are
>>> options to run this in multiple processes. There are two flags that are
>>> of interest:
>>>
>>> 1. NOSE_PROCESS_RESTARTWORKER: - This is documented as follows:
>>> "--process-restartworker
>>>                          If set, will restart each worker process
>>> once their
>>>                          tests are done, this helps control memory leaks
>>> from
>>>                          killing the system.
>>> [NOSE_PROCESS_RESTARTWORKER]"
>>> 2. NOSE_PROCESSES: - This is documented as follows:
>>> "--processes=NUM       Spread test run among this many processes. Set a
>>>                          number equal to the number of processors or
>>> cores in
>>>                          your machine for best results.
>>> [NOSE_PROCESSES]"
>> So - we're moving towards using testr as the test runner _instead_ of
>> nose. There are several reasons for this, but one of them is better
>> support for parallelism.
>>
>> clarkb has been working on this for quantum - you might want to sync up
>> with him if you're interested in helping out.
> 
> Thanks. I have been in touch with Clark. If I understand correctly when
> he ran parallel tests this did not reduce the amount of memory consumed.
> This was the reason I tried to set the above flags. I was hoping for a
> magic bullet.

AH - gotcha. Yeah, magic bullet would be nice.

>>> The problem that I have is that when the first variable is set nothing
>>> is done, this is really dependent on the second. The problem when
>>> setting the second is that I get the following exception:
>>>
>>> garyk at linus:~/quantum$ tox
>>> GLOB sdist-make: /home/garyk/quantum/setup.py
>>> py26 create: /home/garyk/quantum/.tox/py26
>>> ERROR: InterpreterNotFound: python2.6
>>> py27 sdist-reinst: /home/garyk/quantum/.tox/dist/quantum-2013.1.zip
>>> py27 runtests: commands[0]
>>>
>>> /home/garyk/quantum/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/ext/declarative.py:1343:
>>>
>>> SAWarning: The classname 'NetworkBinding' is already in the registry of
>>> this declarative base, mapped to<class
>>> 'quantum.plugins.openvswitch.ovs_models_v2.NetworkBinding'>
>>>    _as_declarative(cls, classname, cls.__dict__)
>>>
>>> Traceback (most recent call last):
>>>    File ".tox/py27/bin/nosetests", line 9, in<module>
>>>      load_entry_point('nose==1.2.1', 'console_scripts', 'nosetests')()
>>>    File
>>> "/home/garyk/quantum/.tox/py27/local/lib/python2.7/site-packages/nose/core.py",
>>>
>>> line 118, in __init__
>>>      **extra_args)
>>>    File "/usr/lib/python2.7/unittest/main.py", line 95, in __init__
>>>      self.runTests()
>>>    File
>>> "/home/garyk/quantum/.tox/py27/local/lib/python2.7/site-packages/nose/core.py",
>>>
>>> line 197, in runTests
>>>      result = self.testRunner.run(self.test)
>>>    File
>>> "/home/garyk/quantum/.tox/py27/local/lib/python2.7/site-packages/nose/plugins/multiprocess.py",
>>>
>>> line 357, in run
>>>      testQueue = Queue()
>>>    File "/usr/lib/python2.7/multiprocessing/managers.py", line 667,
>>> in temp
>>>      token, exp = self._create(typeid, *args, **kwds)
>>>    File "/usr/lib/python2.7/multiprocessing/managers.py", line 565, in
>>> _create
>>>      conn = self._Client(self._address, authkey=self._authkey)
>>>    File "/usr/lib/python2.7/multiprocessing/connection.py", line 175, in
>>> Client
>>>      answer_challenge(c, authkey)
>>>    File "/usr/lib/python2.7/multiprocessing/connection.py", line 413, in
>>> answer_challenge
>>>      message = connection.recv_bytes(256)         # reject large message
>>> IOError: [Errno 11] Resource temporarily unavailable
>>>
>>> Has anyone ever encountered something like this.
>>> Thanks
>>> Gary
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>