Open Stack

On 7/11/14, 7:26 PM, Carl Baldwin wrote:
>
>
> On Jul 11, 2014 5:32 PM, "Vishvananda Ishaya" <vishvananda at gmail.com
<mailto:vishvananda at gmail.com>> wrote:
> >
> > I have tried using pymysql in place of mysqldb and in real world
concurrency
> > tests against cinder and nova it performs slower. I was inspired by
the mention
> > of mysql-connector so I just tried that option instead.
Mysql-connector seems
> > to be slightly slower as well, which leads me to believe that the
blocking inside of
>
> Do you have some numbers?  "Seems to be slightly slower" doesn't
really stand up as an argument against the numbers that have been posted
in this thread.
>
> > sqlalchemy is not the main bottleneck across projects.
> >
> > Vish
> >
> > P.S. The performanace in all cases was abysmal, so performance work
definitely
> > needs to be done, but just the guess that replacing our mysql
library is going to
> > solve all of our performance problems appears to be incorrect at
first blush.
>
> The motivation is still mostly deadlock relief but more performance
work should be done.  I agree with you there.  I'm still hopeful for
some improvement from this.


To identify performance that's alleviated by async you have to establish
up front that IO blocking is the issue, which would entail having code
that's blazing fast until you start running it against concurrent
connections, at which point you can identify via profiling that IO
operations are being serialized.   This is a very specific issue.

In contrast, to identify why some arbitrary openstack app is slow, my
bet is that async is often not the big issue.   Every day I look at
openstack code and talk to people working on things,  I see many
performance issues that have nothing to do with concurrency, and as I
detailed in my wiki page at
https://wiki.openstack.org/wiki/OpenStack_and_SQLAlchemy there is a long
road to cleaning up all the excessive queries, hundreds of unnecessary
rows and columns being pulled over the network, unindexed lookups,
subquery joins, hammering of Python-intensive operations (often due to
the nature of OS apps as lots and lots of tiny API calls) that can be
cached.   There's a clear path to tons better performance documented
there and most of it is not about async  - which means that successful
async isn't going to solve all those issues.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140711/0bbfc250/attachment.html>