Open Stack

On 8/11/2014 5:29 PM, Eoghan Glynn wrote:
>
>> On 8/11/2014 4:22 PM, Eoghan Glynn wrote:
>>>> Hi Eoghan,
>>>>
>>>> Thanks for the note below. However, one thing the overview below does not
>>>> cover is why InfluxDB ( http://influxdb.com/ ) is not being leveraged.
>>>> Many
>>>> folks feel that this technology is a viable solution for the problem space
>>>> discussed below.
>>> Great question Brad!
>>>
>>> As it happens we've been working closely with Paul Dix (lead
>>> developer of InfluxDB) to ensure that this metrics store would be
>>> usable as a backend driver. That conversation actually kicked off
>>> at the Juno summit in Atlanta, but it really got off the ground
>>> at our mid-cycle meet-up in Paris on in early July.
>> ...
>>> The InfluxDB folks have committed to implementing those features in
>>> over July and August, and have made concrete progress on that score.
>>>
>>> I hope that provides enough detail to answer to your question?
>> I guess it begs the question, if influxdb will do what you want and it's
>> open source (MIT) as well as commercially supported, how does gnocchi
>> differentiate?
> Hi Sandy,
>
> One of the ideas behind gnocchi is to combine resource representation
> and timeseries-oriented storage of metric data, providing an efficient
> and convenient way to query for metric data associated with individual
> resources.

Doesn't InfluxDB do the same?

>
> Also, having an API layered above the storage driver avoids locking in
> directly with a particular metrics-oriented DB, allowing for the
> potential to support multiple storage driver options (e.g. to choose
> between a canonical implementation based on Swift, an InfluxDB driver,
> and an OpenTSDB driver, say).
Right, I'm not suggesting to remove the storage abstraction layer. I'm
just curious what gnocchi does better/different than InfluxDB?

Or, am I missing the objective here and gnocchi is the abstraction layer
and not an influxdb alternative? If so, my apologies for the confusion.

> A less compelling reason would be to provide a well-defined hook point
> to innovate with aggregation/analytic logic not supported natively
> in the underlying drivers (e.g. period-spanning statistics such as
> exponentially-weighted moving average or even Holt-Winters).
> Cheers,
> Eoghan
>
>  
>>> Cheers,
>>> Eoghan
>>>
>>>> Thanks,
>>>>
>>>> Brad
>>>>
>>>>
>>>> Brad Topol, Ph.D.
>>>> IBM Distinguished Engineer
>>>> OpenStack
>>>> (919) 543-0646
>>>> Internet: btopol at us.ibm.com
>>>> Assistant: Kendra Witherspoon (919) 254-0680
>>>>
>>>>
>>>>
>>>> From: Eoghan Glynn <eglynn at redhat.com>
>>>> To: "OpenStack Development Mailing List (not for usage questions)"
>>>> <openstack-dev at lists.openstack.org>,
>>>> Date: 08/06/2014 11:17 AM
>>>> Subject: [openstack-dev] [tc][ceilometer] Some background on the gnocchi
>>>> project
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Folks,
>>>>
>>>> It's come to our attention that some key individuals are not
>>>> fully up-to-date on gnocchi activities, so it being a good and
>>>> healthy thing to ensure we're as communicative as possible about
>>>> our roadmap, I've provided a high-level overview here of our
>>>> thinking. This is intended as a precursor to further discussion
>>>> with the TC.
>>>>
>>>> Cheers,
>>>> Eoghan
>>>>
>>>>
>>>> What gnocchi is:
>>>> ===============
>>>>
>>>> Gnocchi is a separate, but related, project spun up on stackforge
>>>> by Julien Danjou, with the objective of providing efficient
>>>> storage and retrieval of timeseries-oriented data and resource
>>>> representations.
>>>>
>>>> The goal is to experiment with a potential approach to addressing
>>>> an architectural misstep made in the very earliest days of
>>>> ceilometer, specifically the decision to store snapshots of some
>>>> resource metadata alongside each metric datapoint. The core idea
>>>> is to move to storing datapoints shorn of metadata, and instead
>>>> allow the resource-state timeline to be reconstructed more cheaply
>>>> from much less frequently occurring events (e.g. instance resizes
>>>> or migrations).
>>>>
>>>>
>>>> What gnocchi isn't:
>>>> ==================
>>>>
>>>> Gnocchi is not a large-scale under-the-radar rewrite of a core
>>>> OpenStack component along the lines of keystone-lite.
>>>>
>>>> The change is concentrated on the final data-storage phase of
>>>> the ceilometer pipeline, so will have little initial impact on the
>>>> data-acquiring agents, or on transformation phase.
>>>>
>>>> We've been totally open at the Atlanta summit and other forums
>>>> about this approach being a multi-cycle effort.
>>>>
>>>>
>>>> Why we decided to do it this way:
>>>> ================================
>>>>
>>>> The intent behind spinning up a separate project on stackforge
>>>> was to allow the work progress at arms-length from ceilometer,
>>>> allowing normalcy to be maintained on the core project and a
>>>> rapid rate of innovation on gnocchi.
>>>>
>>>> Note that that the developers primarily contributing to gnocchi
>>>> represent a cross-section of the core team, and there's a regular
>>>> feedback loop in the form of a recurring agenda item at the
>>>> weekly team meeting to avoid the effort becoming silo'd.
>>>>
>>>>
>>>> But isn't re-architecting frowned upon?
>>>> ======================================
>>>>
>>>> Well, the architecture of other OpenStack projects have also
>>>> under-gone change as the community understanding of the
>>>> implications of prior design decisions has evolved.
>>>>
>>>> Take for example the move towards nova no-db-compute & the
>>>> unified-object-model in order to address issues in the nova
>>>> architecture that made progress towards rolling upgrades
>>>> unneccessarily difficult.
>>>>
>>>> The point, in my understanding, is not to avoid doing the
>>>> course-correction where it's deemed necessary. Rather, the
>>>> principle is more that these corrections happen in an open
>>>> and planned way.
>>>>
>>>>
>>>> The path forward:
>>>> ================
>>>>
>>>> A subset of the ceilometer community will continue to work on
>>>> gnocchi in parallel with the ceilometer core over the remainder
>>>> of the Juno cycle and into the Kilo timeframe. The goal is to
>>>> have an initial implementation of gnocchi ready for tech preview
>>>> by the end of Juno, and to have the integration/migration/
>>>> co-existence questions addressed in Kilo.
>>>>
>>>> Moving the ceilometer core to using gnocchi will be contingent
>>>> on it demonstrating the required performance characteristics and
>>>> providing the semantics needed to support a v3 ceilometer API
>>>> that's fit-for-purpose.
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>