Open Stack

Julien Danjou wrote on 2013-01-23:
> On Wed, Jan 23 2013, Lu, Lianhao wrote:
> 
>> Hi guys,
>> 
>> If my understanding is correct, the current resource_metadata stored by the ceilometer collector could be any python object which can
> be serialized to the JSON format, so the resource_metadata could be a dict containing values of dict/list/string, e.g.
> 
>> 
>>   {"key1": [
>>           { "key11": 11},
>>           { "key12": 12}
>>          ],
>>    "key2": "v2"
>>    "key3": {
>>         "key31": 31
>>        }
>>   }
>> Does the metaquery support complex conditions, like the form of
>> metadata["key3"]["key31"] > 30 or { "key11": 11} in metadata["key1"],
>> besides the simple form of metadata["key2"]=="v2"?
> 
> Nop. The plan for now is to limit metadata querying to key = value where
> value is of a simple type (string, int). I think it's a sane limitation,
> and that will allow us to index more efficiently metadata.
> 
>> Besides, supporting the metaquery in sqlalchemy is not easy as that in
>> mongodb. The question is where should the query filter operation
>> happen? We may do the metadata query filter operation on the
>> sqlalchemy client side in python, which is comparably easy to
>> implement and easy to be adapted to complex filter conditions, but is
>> bad in performance because it needs to get all the records from
>> backend DB to do that.
> 
> We discussed this with Nick back then, and IIRC the idea is to add a
> metadata table with colums 'meter_id', 'key' and 'value' and to index
> those last fields.
> If we were using PG I'd suggest to try arrays or PL/v8 to do JSON
> filtering, but since we have to support MySQL that's not an option I
> guess. D'oh.

I guess what you mentioned here is the table definition in 
http://wiki.openstack.org/Ceilometer/blueprints/multi-dimensions.

One thing concerns me is that it can only handle the first level key/pair
in metadata. What about the other level keypair, e.g. {"key3": {"key31": 31}}?
Should we flatten the metadata into {"key3.key31": 31} and store that into the
metadata table? 

What's more worse is what if the metadata looks like the following? 
  {"key1": [ { "key11": 11}, { "key12": 12} ] }

Or should we put some restriction on the metadata that it must be key/value
dict, in which the value could only be sting/int/float or another dict abiding the same
rules?

-Lianhao
> 
>> The other way is to do the metadata query filter operation on the
>> backend DB side which should have better performance, but I don't know
>> how to do that to support complex conditions other than the simple
>> form of metadata["key2"]=="v2".
> 
> You got the solution: we don't allow that. :)
>