Open Stack

IMHO it's ok and even very natural to expect "escaped" query from users.
e.g, we store the following structure

{metadata:
                    { Zoo:
                               {Foo.Boo: ''value"}}}

and query should be "metadata.Zoo.Foo\.Boo" .
In this case it's not necessary to know deep of tree.

Thanks,
Nadya



On Fri, Aug 29, 2014 at 3:21 PM, Igor Degtiarov <idegtiarov at mirantis.com>
wrote:

> Hi, folks.
>
> I was interested in the problem with storing of samples, that contain
> complex resource_metadata, in MongoDB database [1].
>
> If data is a dict that has a  key(s) with dots (i.e. .), dollar signs
> (i.e. $), or null characters,
> it wouldn't be stored. It is happened because these characters are
> restricted to use in fields name in MongoDB [2], but so far there is no any
> verification of the metadata in ceilometers mongodb driver, as a result we
> will lose data.
>
> Solution of this problem seemed to be rather simple, before storing data
> we check keys in resourse_metadata, if it is a dict, and "quote" keys with
> restricted characters in a similar way, as it was done in a change request
> of redesign separators in columns in HBase [2]. After that store metering
> data.
>
> But other unexpected difficulties appear on the step of getting data. To
> get stored data we constructs a meta query, and structure of that query was
> chosen identical to initial query in MongoDB. So dots is used as a
> separator for three nods of stored data.
>
> Ex. If it is needed to check value in field "Foo"
>
> {metadata:
>                     { Zoo:
>                                {Foo: ''value"}}}
>
> query would be: "metadata.Zoo.Foo"
>
> We don't know how deep is dict in metadata, so it is impossible to propose
> any correct parsing of query, to "quote" field names contain dots.
>
> I see two way for improvements. First is rather complex and based of
> redesign structure of the metadata query in ceilometer. Don't know if it is
> ever possible.
>
> And second is based on removing from the samples "bad" resource_metadata.
> In this case we also lose metadata,  but save other metering data. Of
> course queries for not saved metadata will return nothing, so it is not
> complete solution, but some kind of the hook.
>
> What do you think about that?
> Any thoughts and propositions are kindly welcome.
>
> [1] https://bugs.launchpad.net/mos/+bug/1360240
> [2] http://docs.mongodb.org/manual/reference/limits/
> [3] https://review.openstack.org/#/c/106376/
>
> -- Igor Degtiarov
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140904/a23a49c2/attachment.html>