Open Stack

> On 8/11/2014 4:22 PM, Eoghan Glynn wrote:
> >
> >> Hi Eoghan,
> >>
> >> Thanks for the note below. However, one thing the overview below does not
> >> cover is why InfluxDB ( http://influxdb.com/ ) is not being leveraged.
> >> Many
> >> folks feel that this technology is a viable solution for the problem space
> >> discussed below.
> > Great question Brad!
> >
> > As it happens we've been working closely with Paul Dix (lead
> > developer of InfluxDB) to ensure that this metrics store would be
> > usable as a backend driver. That conversation actually kicked off
> > at the Juno summit in Atlanta, but it really got off the ground
> > at our mid-cycle meet-up in Paris on in early July.
> ...
> >
> > The InfluxDB folks have committed to implementing those features in
> > over July and August, and have made concrete progress on that score.
> >
> > I hope that provides enough detail to answer to your question?
> 
> I guess it begs the question, if influxdb will do what you want and it's
> open source (MIT) as well as commercially supported, how does gnocchi
> differentiate?

Hi Sandy,

One of the ideas behind gnocchi is to combine resource representation
and timeseries-oriented storage of metric data, providing an efficient
and convenient way to query for metric data associated with individual
resources.

Also, having an API layered above the storage driver avoids locking in
directly with a particular metrics-oriented DB, allowing for the
potential to support multiple storage driver options (e.g. to choose
between a canonical implementation based on Swift, an InfluxDB driver,
and an OpenTSDB driver, say).

A less compelling reason would be to provide a well-defined hook point
to innovate with aggregation/analytic logic not supported natively
in the underlying drivers (e.g. period-spanning statistics such as
exponentially-weighted moving average or even Holt-Winters).

Cheers,
Eoghan

 
> > Cheers,
> > Eoghan
> >
> >> Thanks,
> >>
> >> Brad
> >>
> >>
> >> Brad Topol, Ph.D.
> >> IBM Distinguished Engineer
> >> OpenStack
> >> (919) 543-0646
> >> Internet: btopol at us.ibm.com
> >> Assistant: Kendra Witherspoon (919) 254-0680
> >>
> >>
> >>
> >> From: Eoghan Glynn <eglynn at redhat.com>
> >> To: "OpenStack Development Mailing List (not for usage questions)"
> >> <openstack-dev at lists.openstack.org>,
> >> Date: 08/06/2014 11:17 AM
> >> Subject: [openstack-dev] [tc][ceilometer] Some background on the gnocchi
> >> project
> >>
> >>
> >>
> >>
> >>
> >> Folks,
> >>
> >> It's come to our attention that some key individuals are not
> >> fully up-to-date on gnocchi activities, so it being a good and
> >> healthy thing to ensure we're as communicative as possible about
> >> our roadmap, I've provided a high-level overview here of our
> >> thinking. This is intended as a precursor to further discussion
> >> with the TC.
> >>
> >> Cheers,
> >> Eoghan
> >>
> >>
> >> What gnocchi is:
> >> ===============
> >>
> >> Gnocchi is a separate, but related, project spun up on stackforge
> >> by Julien Danjou, with the objective of providing efficient
> >> storage and retrieval of timeseries-oriented data and resource
> >> representations.
> >>
> >> The goal is to experiment with a potential approach to addressing
> >> an architectural misstep made in the very earliest days of
> >> ceilometer, specifically the decision to store snapshots of some
> >> resource metadata alongside each metric datapoint. The core idea
> >> is to move to storing datapoints shorn of metadata, and instead
> >> allow the resource-state timeline to be reconstructed more cheaply
> >> from much less frequently occurring events (e.g. instance resizes
> >> or migrations).
> >>
> >>
> >> What gnocchi isn't:
> >> ==================
> >>
> >> Gnocchi is not a large-scale under-the-radar rewrite of a core
> >> OpenStack component along the lines of keystone-lite.
> >>
> >> The change is concentrated on the final data-storage phase of
> >> the ceilometer pipeline, so will have little initial impact on the
> >> data-acquiring agents, or on transformation phase.
> >>
> >> We've been totally open at the Atlanta summit and other forums
> >> about this approach being a multi-cycle effort.
> >>
> >>
> >> Why we decided to do it this way:
> >> ================================
> >>
> >> The intent behind spinning up a separate project on stackforge
> >> was to allow the work progress at arms-length from ceilometer,
> >> allowing normalcy to be maintained on the core project and a
> >> rapid rate of innovation on gnocchi.
> >>
> >> Note that that the developers primarily contributing to gnocchi
> >> represent a cross-section of the core team, and there's a regular
> >> feedback loop in the form of a recurring agenda item at the
> >> weekly team meeting to avoid the effort becoming silo'd.
> >>
> >>
> >> But isn't re-architecting frowned upon?
> >> ======================================
> >>
> >> Well, the architecture of other OpenStack projects have also
> >> under-gone change as the community understanding of the
> >> implications of prior design decisions has evolved.
> >>
> >> Take for example the move towards nova no-db-compute & the
> >> unified-object-model in order to address issues in the nova
> >> architecture that made progress towards rolling upgrades
> >> unneccessarily difficult.
> >>
> >> The point, in my understanding, is not to avoid doing the
> >> course-correction where it's deemed necessary. Rather, the
> >> principle is more that these corrections happen in an open
> >> and planned way.
> >>
> >>
> >> The path forward:
> >> ================
> >>
> >> A subset of the ceilometer community will continue to work on
> >> gnocchi in parallel with the ceilometer core over the remainder
> >> of the Juno cycle and into the Kilo timeframe. The goal is to
> >> have an initial implementation of gnocchi ready for tech preview
> >> by the end of Juno, and to have the integration/migration/
> >> co-existence questions addressed in Kilo.
> >>
> >> Moving the ceilometer core to using gnocchi will be contingent
> >> on it demonstrating the required performance characteristics and
> >> providing the semantics needed to support a v3 ceilometer API
> >> that's fit-for-purpose.
> >>
> >> _______________________________________________
> >> OpenStack-dev mailing list
> >> OpenStack-dev at lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >>
> >>
> >> _______________________________________________
> >> OpenStack-dev mailing list
> >> OpenStack-dev at lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>