Open Stack

On 4/13/15 8:54 AM, Jeremy Stanley wrote:
> On 2015-04-13 04:03:49 -0400 (-0400), Victor Stinner wrote:
>> Great. Did you notice a performance regression?
> Nope. Worth noting, we implemented it primarily for its lack of
> compiled extensions, and to a lesser because it supports Python 3.x.
> I suspect if we do later run into any unexpected performance
> issues... well, it's pure Python. We have lots of people who can
> help.
>
>> Mike wrote that PyMySQL is much slower than MySQL-Python.

right so it's all a question of proportionality.     First off, some 
major performance improvements have been merged into PyMySQL since I 
wrote my wiki entry on it.   Secondly, one thing I didn't do in that 
wiki (the one here https://wiki.openstack.org/wiki/PyMySQL_evaluation) 
do a performance test of MySQL-Python and PyMySQL talking to a 
non-localhost database; as you can see in the RunSnake profiles I have 
at 
http://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/ 
the proportion of network takes up a lot more over the wire, and when 
you compare MySQL-Python to PyMySQL in that sense the differences in 
Python overhead vs. not become much less a proportion of the overhead 
already.

Take both drivers and put them way deep into the very elaborate 
operations we see Openstack DB backends doing and it's quite difficult 
to observe differences qualitatively; unless you really did speed tests 
for many thousands of identical operations on both.   It's usually in 
the "we tested how long it would take to do 10K or 100K rows / 
operations / etc" space that differences like these add up.     Which is 
why, as is in my blog post, I take the stance that using things like 
eventlet / gevent / async for database code in the first place is not 
really worth the extra headaches at all, even if they did prove to be 
faster than traditional threads, which also I show that they are in fact 
not, for CRUD-style database code as we do in Openstack apps.

> I don't recall him saying that specifically. Also last I recall he
> admitted he hadn't actually tested under the sorts of load we would
> drive in a production OpenStack service--merely performed some
> fairly artificial benchmarks looping known-expensive operations that
> may not ultimately reflect places in our codebase where introducing
> any sort of slowdown would be noticeable compared to other
> operations being performed.
>
> Chances are the Project Infrastructure systems will continue
> incrementally switching to PyMySQL mainly because it's easier to
> install and works on a broader variety of platforms.