Open Stack

On Fri, Mar 21, 2014 at 7:04 AM, Sean Dague <sean at dague.net> wrote:

> On 03/20/2014 06:18 PM, Joe Gordon wrote:
> >
> >
> >
> > On Thu, Mar 20, 2014 at 3:03 PM, Alexei Kornienko
> > <alexei.kornienko at gmail.com <mailto:alexei.kornienko at gmail.com>> wrote:
> >
> >     Hello,
> >
> >     We've done some profiling and results are quite interesting:
> >     during 1,5 hour ceilometer inserted 59755 events (59755 calls to
> >     record_metering_data)
> >     this calls resulted in total 2591573 SQL queries.
> >
> >     And the most interesting part is that 291569 queries were ROLLBACK
> >     queries.
> >     We do around 5 rollbacks to record a single event!
> >
> >     I guess it means that MySQL backend is currently totally unusable in
> >     production environment.
> >
> >
> > It should be noticed that SQLAlchemy is horrible for performance, in
> > nova we usually see sqlalchemy overheads of well over 10x (time
> > nova.db.api call vs the time MySQL measures when slow log is recording
> > everything).
>
> That's not really a fair assessment. Python object inflation takes time.
> I do get that there is SQLA overhead here, but even if you trimmed it
> out you would not get the the mysql query time.
>
> That being said, having Ceilometer's write path be highly tuned and not
> use SQLA (and written for every back end natively) is probably appropriate.
>

I have been working to get Mike Bayer (author of SQLAlchemy) to the summit
in Atlanta. He is interested in working with us to improve SQLAlchemy, so
if we have specific performance or feature issues like this, it would be
good to make a list. If we have enough, maybe we can set aside a session
in the Oslo track, otherwise we can at least have some hallway
conversations.

Doug



>
>         -Sean
>
> --
> Sean Dague
> Samsung Research America
> sean at dague.net / sean.dague at samsung.com
> http://dague.net
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140321/f008ebe0/attachment.html>