Open Stack

Victor,

We're aware that there are ways to mock (and un-mock) correctly.  We're trying to make sure that all our new test code follows those patterns.  We also decided that it wouldn't make sense to change all the working tests to use the 'right' methods as that could have the short-term effect of destabilizing the tests considerably (plus it would be a fair bit of effort with very little actual gain). As a compromise we added this code to check that things weren't left mocked (just in case someone copy-pasted the old style, did it wrong and no-one noticed - we're still trying to maintain consistency wherever possible).  As the code base gets cleaned up we can eventually run this just once per test class, or even just once at the end of the test run.  Petr has a changeset up now that will reduce it to twice per class (https://review.openstack.org/#/c/312781 ), and as I said we can drop that to just one probably by the end of the cycle.

Amrith is right though - python34 is *much* slower running this check (and somewhat in general) than python27.  Maybe that doesn't translate into real-world performance data, but it has made me a bit nervous nonetheless.

Peter


-----Original Message-----
From: Victor Stinner [mailto:vstinner at redhat.com] 
Sent: May-16-16 9:38 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [trove] timeouts in gate-trove-python34-db

Le 16/05/2016 13:52, Amrith Kumar a écrit :
>> IMHO the strange mock detector code must be removed. It is very slow 
>> and I don't understand its purpose.
>
> [amrith] It serves and has served a very useful purpose and that is to 
> detect bad tests where code has (and we've had lots of trouble with 
> this) established a Mock() but failed to delete it.

There are many options to disable (unregister, stop, call it as you
want) mocks automatically. As I wrote, fixtures & oslotest give you tools to do that automatically. It's also a base feature of the mock
module: "with mock.patch(...): ...". Sorry, I don't know enough the Trove code base (code of the unit tests) to say which option is the best.


> I'd rather figure out why it is slower in Python3 because it may be 
> indicative of something that may impact other parts of the code as 
> well. We're having this whole discussion about Python performance and 
> the Go language, I think it is not a good idea to delete code which is 
> performing poorly because it is performing slowly.

Sorry but Go doesn't solve badly designed functions :-) It's an algorithmic complexity issue: the function has to iterate on *all* alive Python objects: complexity of O(n). I propose to remove to code to simplify the complexity to O(1) :-)

Ok, let's take an example with Python 2.7: the command "python -bb -m testtools.run trove/tests/unittests/common/test_exception.py" takes 972 ms. Remove the mock workaround, the command now taks 1 ms: it's now 972x faster.

=> removing the slow workaround makes test ~1000x faster!

IMHO it's worth to start here, instead of starting to investigate why running tests on Python 3 seems slower.

I'm not aware of such huge performance difference between Python 2 and Python 3 when running unit tests on other OpenStack services. The issue is specific to Trove, and IMHO it comes from the mock workaround.

Victor

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev