Open Stack

Hi Julien,

Been working a bit on this and have a patch based on master that is working at:

https://github.com/NeCTAR-RC/gnocchi/tree/influxdb-driver

I could push it up to gerrit but I think something will need to change for it to run the influxdb tests?

It should act more like the carbonara drivers now as opposed to the old influx driver. It will do downsampling and retention based on the archive policies.

Currently it is failing one test [1] and that is to do with retention. 
This is because influxDB does retention based on the current time, e.g. a 1 day retention policy will be from the current time. 
The tests assume that the retention period is based on the data stored and so it will keep 1 day of data no matter how old that data is.

I also had to disable retention policies in influx while running the tests as when I backfill data influx is too smart and won’t backfill data that wouldn’t meet the retention policy.
One way to fix all this would be to change all the test times to be relative from now but then there could be other race conditions etc. I think.

I’m still not 100% happy with the code, particularly around how the continuous queries are created based on the archive policies.

We are using this code in preprod and so far all is going well. 

Should also note that you will need influxDB v1.0 to use this. To test all you should need is an influx server running locally.

Would love some feed back particularly from people who know how influx works as there are a few ways it could be architected.

Cheers,
Sam

[1] gnocchi.tests.test_rest.MetricTest.test_add_measures_back_window




> On 6 Sep 2016, at 11:24 AM, Sam Morrison <sorrison at gmail.com> wrote:
> 
> Hi Julien,
> 
>> On 5 Sep 2016, at 5:36 PM, Julien Danjou <julien at danjou.info> wrote:
>> 
>> On Mon, Sep 05 2016, Sam Morrison wrote:
>> 
>> Hi Sam,
>> 
>>> The issue I’m having are with the tests. Because the continuous queries are
>>> asynchronous and there is no current way in influxdb to force them to run I get
>>> tests failing due to
>>> them not having run yet.
>>> 
>>> I’m not sure how to get around this issue, apart from the tests failing
>>> everything is working quite well. I’m going to start some load testing soon to
>>> see what it’s like when pushing in a lot of metrics.
>> 
>> Does it break the REST API, or only some storage tests?
> 
> REST API is fine, in fact it fixes some tests that influx was failing on.
> 
>> If it's just some storage test, you can change the tests so they are
>> retrying until the operation are done. Either in the test, or via a
>> special flag in the driver – we used to have that in the first version
>> of the driver.
> 
> OK good idea, I’ll work on that.
> 
> 
>>> Wondering if there would be time to talk about this in Barcelona.
>> 
>> Sure.
> 
> Cheers,
> Sam
> 
> 
>> 
>> -- 
>> Julien Danjou
>> -- Free Software hacker
>> -- https://julien.danjou.info