Open Stack

On 8/14/13 3:26 AM, "Julien Danjou" <julien at danjou.info> wrote:

>On Tue, Aug 13 2013, Thomas Maddox wrote:
>
>> I was curious about why we went for a JOIN here rather than just using
>>the
>> meter table initially?
>> 
>>https://github.com/openstack/ceilometer/blob/master/ceilometer/storage/im
>>pl_sqlalchemy.py#L336-L391.
>> Doug had mentioned that some performance testing had gone on with some
>>of
>> these queries, so before writing up requests to change this to the meter
>> table only, I wanted to see if this was a result of that performance
>> testing? Like the JOIN was less expensive than a DISTINCT.
>
>Because the SQL driver has been historically a straight conversion from
>MongoDB, who was working that way.
>The MongoDB driver had these 2 collections, meter and resources, and
>used both of them to construct these data.
>
>The original idea was to register resources in the eponym table, and use
>it to filter. Unfortunately, that didn't work out, especially with APIv2
>that allows to filter on many things. Among other things, there was
>timestamp fields in the resource table that were used to filter based on
>timestamp, but this was a naive implementation that failed pretty soon,
>and we removed and replace with a more solid one based on the meter
>table.
>
>What you see here are the rest of that time, and in the end, this
>resource table should be removed AFAICT, in MongoDB also.

Ahhh yep, when I look at the two side-by-side I can see what you're
saying. Thanks for the explanation!

So, then should I write up a BP for improvements of this sort, like the
API improvement one? It seems like we may get better results if the SQL
implementation is more SQL-like than Mongo-like since they are
fundamentally different things. I noticed some notes in there also about
the JOIN being especially expensive.

>
>-- 
>Julien Danjou
>;; Free Software hacker ; freelance consultant
>;; http://julien.danjou.info