Open Stack

Hi Julien,

I’ve been working on the influx driver and have been re designing it to use continuous queries and retention policies so it acts more like the carbonara based drivers.
Basically the continuous queries down sample the data. All metrics are stored in the same influx measurement with the metric_id being a tag.

The issue I’m having are with the tests. Because the continuous queries are asynchronous and there is no current way in influxdb to force them to run I get tests failing due to
them not having run yet.

I’m not sure how to get around this issue, apart from the tests failing everything is working quite well. I’m going to start some load testing soon to see what it’s like when pushing in a lot of metrics.

Wondering if there would be time to talk about this in Barcelona.

Cheers,
Sam



> On 4 Aug 2016, at 2:59 PM, Sam Morrison <sorrison at gmail.com> wrote:
> 
> OK thanks Julien,
> 
> I’m about to go on holiday for a month so I’ll pick this up when I return. One of our devs is playing with this and thinking of ways to support the things currently not implemented/working.
> 
> Cheers,
> Sam
> 
> 
>> On 2 Aug 2016, at 8:35 PM, Julien Danjou <julien at danjou.info> wrote:
>> 
>> On Tue, Aug 02 2016, Sam Morrison wrote:
>> 
>> Hi Sam!
>> 
>>> We have been using gnocchi for a while now with the influxDB driver
>>> and are keen to get the influxdb driver back into upstream.
>>> 
>>> However looking into the code and how it’s arranged it looks like
>>> there are a lot of assumptions that the backend storage driver is
>>> carbonara based.
>> 
>> More or less. There is a separation layer (index/storage) and a full
>> abstraction layer so it's possible to write a driver for any TSDB.
>> Proof, we had an InfluxDB driver.
>> Now the separation layer is not optimal for some TSDBs like InfluxDB,
>> unfortunately nobody never stepped up to enhance it.
>> 
>>> Is gnocchi an API for time series DBs or is it a time series DB
>>> itself?
>> 
>> Both. It's an API over TSDBs, and it also has its own TSDB based on
>> Carbonara+{Ceph,File,Swift}.
>> 
>>> The tests that are failing are due to the way carbonara and influx handle the
>>> retention and multiple granularities differently. (which we can work around
>>> outside of gnocchi for now)
>>> 
>>> So I guess I’m wondering if there will be support for other drivers apart from carbonara?
>> 
>> Sure. We dropped the InfluxDB driver because nobody was maintaining it
>> and it was not passing the tests anymore. But we'd be glad to have it
>> in-tree I'd say.
>> 
>>> We use influx because we already use it for other stuff within our organisation
>>> and don’t want to set up ceph or swift (which is quite an endeavour) to support
>>> another time series DB.
>> 
>> That makes sense. If you don't need scaling, I can only encourage you
>> taking a look at using Carbonara+file rather than InfluxDB in the
>> future, which I think is still a better choice.
>> 
>> But in the meantime, feel free to send a patch to include back InfluxDB
>> in Gnocchi. As long as you're ready to help us maintain it, we'll all
>> open on that. :)
>> 
>> Cheers,
>> -- 
>> Julien Danjou
>> # Free Software hacker
>> # https://julien.danjou.info
>