Open Stack

On 8/11/2014 6:49 PM, Eoghan Glynn wrote:
>
>>>> On 8/11/2014 4:22 PM, Eoghan Glynn wrote:
>>>>>> Hi Eoghan,
>>>>>>
>>>>>> Thanks for the note below. However, one thing the overview below does
>>>>>> not
>>>>>> cover is why InfluxDB ( http://influxdb.com/ ) is not being leveraged.
>>>>>> Many
>>>>>> folks feel that this technology is a viable solution for the problem
>>>>>> space
>>>>>> discussed below.
>>>>> Great question Brad!
>>>>>
>>>>> As it happens we've been working closely with Paul Dix (lead
>>>>> developer of InfluxDB) to ensure that this metrics store would be
>>>>> usable as a backend driver. That conversation actually kicked off
>>>>> at the Juno summit in Atlanta, but it really got off the ground
>>>>> at our mid-cycle meet-up in Paris on in early July.
>>>> ...
>>>>> The InfluxDB folks have committed to implementing those features in
>>>>> over July and August, and have made concrete progress on that score.
>>>>>
>>>>> I hope that provides enough detail to answer to your question?
>>>> I guess it begs the question, if influxdb will do what you want and it's
>>>> open source (MIT) as well as commercially supported, how does gnocchi
>>>> differentiate?
>>> Hi Sandy,
>>>
>>> One of the ideas behind gnocchi is to combine resource representation
>>> and timeseries-oriented storage of metric data, providing an efficient
>>> and convenient way to query for metric data associated with individual
>>> resources.
>> Doesn't InfluxDB do the same?
> InfluxDB stores timeseries data primarily.
>
> Gnocchi in intended to store strongly-typed OpenStack resource
> representations (instance, images, etc.) in addition to providing
> a means to access timeseries data associated with those resources.
>
> So to answer your question: no, IIUC, it doesn't do the same thing.

Ok, I think I'm getting closer on this.  Thanks for the clarification.
Sadly, I have more questions :)

Is this closer? "a metadata repo for resources (instances, images, etc)
+ an abstraction to some TSDB(s)"?

Hmm, thinking out loud ... if it's a metadata repo for resources, who is
the authoritative source for what the resource is? Ceilometer/Gnocchi or
the source service? For example, if I want to query instance power state
do I ask ceilometer or Nova?

Or is it metadata about the time-series data collected for that
resource? In which case, I think most tsdb's have some sort of "series
description" facilities. I guess my question is, what makes this
metadata unique and how would it differ from the metadata ceilometer
already collects?

Will it be using Glance, now that Glance is becoming a pure metadata repo?


> Though of course these things are not a million miles from each
> other, one is just a step up in the abstraction stack, having a
> wider and more OpenStack-specific scope.

Could it be a generic timeseries service? Is it "openstack specific"
because it uses stackforge/python/oslo? I assume the rules and schemas
will be data-driven (vs. hard-coded)? ... and since the ceilometer
collectors already do the bridge work, is it a pre-packaging of
definitions that target openstack specifically? (not sure about "wider
and more specific")

Sorry if this was already hashed out in Atlanta.

>  
>>> Also, having an API layered above the storage driver avoids locking in
>>> directly with a particular metrics-oriented DB, allowing for the
>>> potential to support multiple storage driver options (e.g. to choose
>>> between a canonical implementation based on Swift, an InfluxDB driver,
>>> and an OpenTSDB driver, say).
>> Right, I'm not suggesting to remove the storage abstraction layer. I'm
>> just curious what gnocchi does better/different than InfluxDB?
>>
>> Or, am I missing the objective here and gnocchi is the abstraction layer
>> and not an influxdb alternative? If so, my apologies for the confusion.
> No worries :)
>
> The intention is for gnocchi to provide an abstraction over
> timeseries, aggregation, downsampling and archiving/retention
> policies, with a number of drivers mapping onto real timeseries
> storage options. One of those drivers is based on Swift, another
> is in the works based on InfluxDB, and a third based on OpenTSDB
> has also been proposed.
>
> Cheers,
> Eoghan
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>