Open Stack

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 12/07/14 03:17, Mike Bayer wrote:
> 
> On 7/11/14, 7:26 PM, Carl Baldwin wrote:
>> 
>> 
>> On Jul 11, 2014 5:32 PM, "Vishvananda Ishaya"
>> <vishvananda at gmail.com
> <mailto:vishvananda at gmail.com>> wrote:
>>> 
>>> I have tried using pymysql in place of mysqldb and in real
>>> world
> concurrency
>>> tests against cinder and nova it performs slower. I was
>>> inspired by
> the mention
>>> of mysql-connector so I just tried that option instead.
> Mysql-connector seems
>>> to be slightly slower as well, which leads me to believe that
>>> the
> blocking inside of
>> 
>> Do you have some numbers?  "Seems to be slightly slower" doesn't
> really stand up as an argument against the numbers that have been
> posted in this thread.
>> 
>>> sqlalchemy is not the main bottleneck across projects.
>>> 
>>> Vish
>>> 
>>> P.S. The performanace in all cases was abysmal, so performance
>>> work
> definitely
>>> needs to be done, but just the guess that replacing our mysql
> library is going to
>>> solve all of our performance problems appears to be incorrect
>>> at
> first blush.
>> 
>> The motivation is still mostly deadlock relief but more
>> performance
> work should be done.  I agree with you there.  I'm still hopeful
> for some improvement from this.
> 
> 
> To identify performance that's alleviated by async you have to
> establish up front that IO blocking is the issue, which would
> entail having code that's blazing fast until you start running it
> against concurrent connections, at which point you can identify via
> profiling that IO operations are being serialized.   This is a very
> specific issue.
> 
> In contrast, to identify why some arbitrary openstack app is slow,
> my bet is that async is often not the big issue.   Every day I look
> at openstack code and talk to people working on things,  I see
> many performance issues that have nothing to do with concurrency,
> and as I detailed in my wiki page at 
> https://wiki.openstack.org/wiki/OpenStack_and_SQLAlchemy there is a
> long road to cleaning up all the excessive queries, hundreds of
> unnecessary rows and columns being pulled over the network,
> unindexed lookups, subquery joins, hammering of Python-intensive
> operations (often due to the nature of OS apps as lots and lots of
> tiny API calls) that can be cached.   There's a clear path to tons
> better performance documented there and most of it is not about
> async  - which means that successful async isn't going to solve all
> those issues.
> 

Of course there is a long road to decent performance, and switching a
library won't magically fix all out issues. But if it will fix
deadlocks, and give 30% to 150% performance boost for different
operations, and since the switch is almost smooth, this is something
worth doing.

> 
> 
> 
> _______________________________________________ OpenStack-dev
> mailing list OpenStack-dev at lists.openstack.org 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBCgAGBQJTwrP+AAoJEC5aWaUY1u57gRUH+wTAUcl1kujT5iVUwcZUEEfx
P9nC0JPduXxYlobiFYyQKVVQm6pTaPgbgBoq4M/vKD0PbxBYMSTFA+igmewX6cHN
RlsfvQgTsB/FU+dxK3gfRBQU3OnHLUSKWwZydp0YjmrCCHP3wiPj/HqWdD7vl1H8
Cxl+R2Zr7dR1ZMQBHDbAtu6FrGEa5SBnAwvAcsIr6mKymkFzQ4DU2DKOc8mm1i1f
GhwCG8IvhKYo/w3yt0CUjPJoPSDaAoIGv984NDv7sjHeSZCRxNmV8jXlRUcztFcG
lBAWguZtdyOiX+qHNmRZHV1Mm0SybcGIZN0Zw+u+1hpoiR0ZjAbLoofBwkCHxDE=
=OXs7
-----END PGP SIGNATURE-----