[openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

Boris Pavlovic boris at pavlovic.me
Fri Mar 21 11:29:52 UTC 2014


Sean,


Absolutely agree with you.
It's not the same to execute query and get plain text, and execute query
and get hierarchy of python objects.

Plus I disagree when I hear that SQLAlchemy is slow. It's slow when you are
using it wrong.

Like in Nova Scheduler [1] we were fetching full 3 tables with JOIN. Which
produce much more results from DB (in bytes and rows) then just make 3
separated selects and then join it by hand.

We should stop using next phrases:
1) python is slow
2) mysql is slow
3) sqlalchemy is slow
4) hardware is slow [2]

And start using these phrase:
1) Algorithms that we are using are bad
2) Architecture solutions that we are using are bad

And start thinking about how to improve them.


[1] https://review.openstack.org/#/c/43151/
[2] http://en.wikipedia.org/wiki/Buran_(spacecraft)

Best regards,
Boris Pavlovic



On Fri, Mar 21, 2014 at 3:04 PM, Sean Dague <sean at dague.net> wrote:

> On 03/20/2014 06:18 PM, Joe Gordon wrote:
> >
> >
> >
> > On Thu, Mar 20, 2014 at 3:03 PM, Alexei Kornienko
> > <alexei.kornienko at gmail.com <mailto:alexei.kornienko at gmail.com>> wrote:
> >
> >     Hello,
> >
> >     We've done some profiling and results are quite interesting:
> >     during 1,5 hour ceilometer inserted 59755 events (59755 calls to
> >     record_metering_data)
> >     this calls resulted in total 2591573 SQL queries.
> >
> >     And the most interesting part is that 291569 queries were ROLLBACK
> >     queries.
> >     We do around 5 rollbacks to record a single event!
> >
> >     I guess it means that MySQL backend is currently totally unusable in
> >     production environment.
> >
> >
> > It should be noticed that SQLAlchemy is horrible for performance, in
> > nova we usually see sqlalchemy overheads of well over 10x (time
> > nova.db.api call vs the time MySQL measures when slow log is recording
> > everything).
>
> That's not really a fair assessment. Python object inflation takes time.
> I do get that there is SQLA overhead here, but even if you trimmed it
> out you would not get the the mysql query time.
>
> That being said, having Ceilometer's write path be highly tuned and not
> use SQLA (and written for every back end natively) is probably appropriate.
>
>         -Sean
>
> --
> Sean Dague
> Samsung Research America
> sean at dague.net / sean.dague at samsung.com
> http://dague.net
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140321/be3920ac/attachment.html>


More information about the OpenStack-dev mailing list