Open Stack

On Thu, Jan 24 2013, Sandy Walsh wrote:

> This seems like a very inefficient schema requiring multiple sub-queries.
>
> Other than the naming, is it really any different than the current
> Metadata table when it comes to db performance?

There's no metadata table currently, there's a metadata *column*.
You can't do any filtering request based on that current column in pure
SQL.

> I think a better approach would be to offer different Metric types
> (extensible) which can control their own mapping to the db.

I can't see how you can do that and supports a large amount of different
metrics type, being generic.

But I think that you may want is to implement an dynamic SQL engine
backend creating and indexing columns you want to request for. That's a
solution, but we're trying to be generic with the default sqlalchemy
backend.

> I'd be curious to see how the metadata table approach performs when you
> are querying on multiple keys (like Event Name + Cell + Host + Request
> ID, for example) with a large number of rows. Has anyone tried this?

I don't think someone did. This blueprint draft was just something we
talked about back then with Nick and we wrote some ideas to not forget
it and have some things to discuss.

The problem is that metadata are EAV and that plays badly with SQL (and
especially with SQL lowered down to basics thanks to ORM abstraction and
SQLAlchemy). It's not clear that doing splitting the metadata in another
table is going to be more efficient, even if data are indexed. It may be
faster to use SQL indexes to retrieve matching events as it is, and do
the final metadata filtering at application level (i.e. in
storage.impl_sqlalchemy).

As you said, that should probably be tested.

FTR I've created a blueprint on this:

    https://blueprints.launchpad.net/ceilometer/+spec/sqlalchemy-metadata-query

-- 
Julien Danjou
/* Free Software hacker & freelance
   http://julien.danjou.info */
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130124/c26192d4/attachment.pgp>