Open Stack

On 07/11/2014 09:17 PM, Mike Bayer wrote:
...
> To identify performance that's alleviated by async you have to
> establish up front that IO blocking is the issue, which would entail
> having code that's blazing fast until you start running it against
> concurrent connections, at which point you can identify via profiling
> that IO operations are being serialized.   This is a very specific
> issue.
>
> In contrast, to identify why some arbitrary openstack app is slow, my
>  bet is that async is often not the big issue.   Every day I look at
>  openstack code and talk to people working on things,  I see many
> performance issues that have nothing to do with concurrency, and as I
>  detailed in my wiki page at
> https://wiki.openstack.org/wiki/OpenStack_and_SQLAlchemy there is a
> long road to cleaning up all the excessive queries, hundreds of
> unnecessary rows and columns being pulled over the network, unindexed
> lookups, subquery joins, hammering of Python-intensive operations
> (often due to the nature of OS apps as lots and lots of tiny API
> calls) that can be cached.   There's a clear path to tons better
> performance documented there and most of it is not about async  -
> which means that successful async isn't going to solve all those
> issues.

Yep, couldn't agree more.

Frankly, the steps you outline in the wiki above are excellent examples 
of where we can make significant gains in both performance and 
scalability. In addition to those you listed, the underlying database 
schemas themselves, with the excessive use of large VARCHAR fields, BLOB 
fields for JSONified values, and the general bad strategy of bunching 
heavily-read fields with infrequently-read fields in the same tables, 
are also a source of poor overall database performance.

Best,
-jay