Open Stack

> Hi Eoghan,
> 
> Thanks for the note below. However, one thing the overview below does not
> cover is why InfluxDB ( http://influxdb.com/ ) is not being leveraged. Many
> folks feel that this technology is a viable solution for the problem space
> discussed below.

Great question Brad!

As it happens we've been working closely with Paul Dix (lead
developer of InfluxDB) to ensure that this metrics store would be
usable as a backend driver. That conversation actually kicked off
at the Juno summit in Atlanta, but it really got off the ground
at our mid-cycle meet-up in Paris on in early July.

I wrote a rough strawman version of an InfluxDB driver in advance
of the mid-cycle to frame the discussion, and Paul Dix traveled
to the meet-up so we could have the discussion face-to-face. The
conclusion was that InfluxDB would indeed potentially be a great
fit, modulo some requirements that we identified during the detailed
discussions:

 * shard-space-based retention & backgrounded deletion
 * capability to merge individual timeseries for cross-aggregation
 * backfill-aware downsampling

The InfluxDB folks have committed to implementing those features in
over July and August, and have made concrete progress on that score.

I hope that provides enough detail to answer to your question?

Cheers,
Eoghan

> Thanks,
> 
> Brad
> 
> 
> Brad Topol, Ph.D.
> IBM Distinguished Engineer
> OpenStack
> (919) 543-0646
> Internet: btopol at us.ibm.com
> Assistant: Kendra Witherspoon (919) 254-0680
> 
> 
> 
> From: Eoghan Glynn <eglynn at redhat.com>
> To: "OpenStack Development Mailing List (not for usage questions)"
> <openstack-dev at lists.openstack.org>,
> Date: 08/06/2014 11:17 AM
> Subject: [openstack-dev] [tc][ceilometer] Some background on the gnocchi
> project
> 
> 
> 
> 
> 
> Folks,
> 
> It's come to our attention that some key individuals are not
> fully up-to-date on gnocchi activities, so it being a good and
> healthy thing to ensure we're as communicative as possible about
> our roadmap, I've provided a high-level overview here of our
> thinking. This is intended as a precursor to further discussion
> with the TC.
> 
> Cheers,
> Eoghan
> 
> 
> What gnocchi is:
> ===============
> 
> Gnocchi is a separate, but related, project spun up on stackforge
> by Julien Danjou, with the objective of providing efficient
> storage and retrieval of timeseries-oriented data and resource
> representations.
> 
> The goal is to experiment with a potential approach to addressing
> an architectural misstep made in the very earliest days of
> ceilometer, specifically the decision to store snapshots of some
> resource metadata alongside each metric datapoint. The core idea
> is to move to storing datapoints shorn of metadata, and instead
> allow the resource-state timeline to be reconstructed more cheaply
> from much less frequently occurring events (e.g. instance resizes
> or migrations).
> 
> 
> What gnocchi isn't:
> ==================
> 
> Gnocchi is not a large-scale under-the-radar rewrite of a core
> OpenStack component along the lines of keystone-lite.
> 
> The change is concentrated on the final data-storage phase of
> the ceilometer pipeline, so will have little initial impact on the
> data-acquiring agents, or on transformation phase.
> 
> We've been totally open at the Atlanta summit and other forums
> about this approach being a multi-cycle effort.
> 
> 
> Why we decided to do it this way:
> ================================
> 
> The intent behind spinning up a separate project on stackforge
> was to allow the work progress at arms-length from ceilometer,
> allowing normalcy to be maintained on the core project and a
> rapid rate of innovation on gnocchi.
> 
> Note that that the developers primarily contributing to gnocchi
> represent a cross-section of the core team, and there's a regular
> feedback loop in the form of a recurring agenda item at the
> weekly team meeting to avoid the effort becoming silo'd.
> 
> 
> But isn't re-architecting frowned upon?
> ======================================
> 
> Well, the architecture of other OpenStack projects have also
> under-gone change as the community understanding of the
> implications of prior design decisions has evolved.
> 
> Take for example the move towards nova no-db-compute & the
> unified-object-model in order to address issues in the nova
> architecture that made progress towards rolling upgrades
> unneccessarily difficult.
> 
> The point, in my understanding, is not to avoid doing the
> course-correction where it's deemed necessary. Rather, the
> principle is more that these corrections happen in an open
> and planned way.
> 
> 
> The path forward:
> ================
> 
> A subset of the ceilometer community will continue to work on
> gnocchi in parallel with the ceilometer core over the remainder
> of the Juno cycle and into the Kilo timeframe. The goal is to
> have an initial implementation of gnocchi ready for tech preview
> by the end of Juno, and to have the integration/migration/
> co-existence questions addressed in Kilo.
> 
> Moving the ceilometer core to using gnocchi will be contingent
> on it demonstrating the required performance characteristics and
> providing the semantics needed to support a v3 ceilometer API
> that's fit-for-purpose.
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>