Open Stack

Hi,

Originally, I posted this question on the review [0] that adds InfluxDB
support to Gnocchi but Julien felt that it wasn't relevant in the scope of
the review. Still I think that it deserves some discussion...

The current implementation of the InfluxDB driver for Gnocchi doesn't
follow the recommendations for InfluxDB 0.9 [1] as it doesn't use tags at
all. As a result, each metric will be stored in an individual series which
makes aggregation across metrics suboptimal from the InfluxDB point of
view. With tags properly implemented, a query like 'return the cpu.util
measures for this group of servers in this given interval' is only one
InfluxDB query while it would result in N queries with the proposed change.
In fact, the same "issue" can be seen in the OpenTSDB [2] and KairosDB [3]
reviews too. And my guess is that all production-grade backends will
provide the same type of semantic on metrics (call it tags, labels or
dimensions).

Julien's anwser to this was:

There's no point in talking about optimizing a driver until it's
implemented. For now, neither InfluxDB or Kairos nor OpenTSDB drivers are
ready for Gnocchi. Once they are, we'll be able to talk about changing the
implementation of the storage/driver API to leverage their abilities such
as tags.

I'm still struggling to see how these optimizations would be implemented
since the current Gnocchi design has separate backends for indexing and
storage which means that datapoints (id + timestamp + value) and metric
metadata (tenant_id, instance_id, server group, ...) are stored into
different places. I'd be interested to hear from the Gnocchi team how this
is going to be tackled. For instance, does it imply modifications or
extensions to the existing Gnocchi API?

BR,
Simon

[0] https://review.openstack.org/#/c/165407/
[1] http://influxdb.com/docs/v0.9/concepts/schema_and_data_layout.html
[2] https://review.openstack.org/#/c/107986
[3] https://review.openstack.org/#/c/159476
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150616/f1518cbe/attachment.html>