Open Stack

I think Sensu is the best monitoring approach out there atm. Nagios /
Icinga are way to static and scale badly imho.  The kind of checks you
proposed are quite interesting. I would suggest to wrap a sensu check
around Tempest but that's going to far for the first cycle.

The two stacks (Sensu + Unchiwa and TICK) only really overlap in metrics
collection which can be done via Sensu and Telegraf. I don't know if it
makes sense to have both ... I definitely think we need Sensu though simply
to monitor service availability and other thresholds and events which
aren't covered in TICK as not everything is time series data and to have
the alerting. Only with Sensu we don't have insight into performance and
trends, with TICK only we lack alerting on events and non-performance
metric data (Is Keystone up? etc)

I think it won't hurt to develop theses two stacks in parallel and maybe
we'll join them together in a chain as I described earlier.

2016-07-24 14:25 GMT+02:00 Dave Walker <email at daviey.com>:

> Thanks Mathias,
>
> I'm not tied to Sensu.. anything can really fill that gap in my mind.
> You've done a good job at outlining the steps involved.  I created a
> blueprint with the steps I had in mind[0]
>
> For this cycle, I wanted to keep it simple so it was easily achievable.  I
> only planned to have some basic up/down for each node and throw
> the performance data on the floor.
>
> I wanted to include the option to include local configs, as json blobs.  Some
> of the things I was thinking as local config:
>   - daily checkouts, can instances be built with networking
>   - remaining resources count (ie, does each subnet have X remaining ip
> addresses available)
>   - Is Ceph healthy?
>
> So, these things aren't really performance over time interesting.. which
> means the intention does differ.  However, I do agree that both stacks
> could achieve both objectives.
>
> I've essentially got much of this working locally, but would require about
> a day of cleaning up for submission... but if your work can achieve the
> objectives above, i'm happy to discontinue... or help make your stack
> pluggable.
>
> [0] https://blueprints.launchpad.net/kolla/+spec/sensu
>
> --
> Kind Regards,
> Dave Walker
>
> On 24 July 2016 at 11:56, Mathias Ewald <mewald at evoila.de> wrote:
>
>> Monitoring is a difficult topic as the number of options regarding the
>> toolset and mechanisms are very high. We had some chats about it in IRC
>> that discovered even more options than I thought existed :D I believe
>> Dave's view on Sensu is generally correct in that Sensu is more directed to
>> monitoring in the form of "if X running/working" but of course has the
>> ability to transport metrics, too, but lacks the good dashboarding
>> capabilities for performance data. One set up I could images is
>>
>> 1. Sensu Client to collect checks and metrics
>> 2. RabbitMQ for transport
>> 3. Sensu Server to receive, evaluate, alarm and write metrics to InfluxDB
>> 4. Uchiwa as a Dashboard to Sensu
>> 5. InfluxDB to store metrics
>> 6. Grafana to dashboard metrics
>>
>> So Sensu could be used as a replacement for (or in addition to) a metrics
>> collection daemon like Collectd or what I decided to use: Telegraf. For my
>> implementation, this means I will add a parameter to make Telegraf
>> optional. This way, someone else may implement the rest of the stack and
>> the user can decide which one to use.
>>
>> What do you think?
>>
>> Mathias
>>
>>
>>
>> 2016-07-23 21:51 GMT+02:00 Stephen Hindle <shindle at llnw.com>:
>>
>>> My understanding was Sensu could produce metrics ?
>>> And Kapacitor can do alerting for the TICK stack stuff mewald is doing...
>>> I really don't see them as that different ?
>>>
>>>
>>> On Fri, Jul 22, 2016 at 5:19 PM, Dave Walker <email at daviey.com> wrote:
>>> > Yes, this is my thought.
>>> >
>>> > The scope of the Sensu work is: "Is this thing working?" (with the
>>> reference
>>> > being up/down)
>>> > But the scope of the Grafana and friends is, "How hard is this
>>> working?"
>>> > (but no alerting)
>>> >
>>> > They are certainly complementary.... However, Sensu can throw data at a
>>> > Grafana stack (aiui).. but I fear that is too much to achieve this
>>> cycle.
>>> >
>>> > --
>>> > Kind Regards,
>>> > Dave Walker
>>> >
>>> > On 23 July 2016 at 00:11, Fox, Kevin M <Kevin.Fox at pnnl.gov> wrote:
>>> >>
>>> >> I think those are two different, complementary things.
>>> >>
>>> >> One's metrics and the other is monitoring. You probably want both at
>>> the
>>> >> same time.
>>> >>
>>> >> Thanks,
>>> >> Kevin
>>> >> ________________________________________
>>> >> From: Steven Dake (stdake) [stdake at cisco.com]
>>> >> Sent: Friday, July 22, 2016 3:52 PM
>>> >> To: OpenStack Development Mailing List (not for usage questions)
>>> >> Subject: Re: [openstack-dev] [kolla] Monitoring tooling
>>> >>
>>> >> Thanks for pointing that out.  Brain out to lunch today it appears :(
>>> >>
>>> >> I think choices are a good thing even though they increase our
>>> >> implementation footprint.  Anyone opposed to implementing both with
>>> >> something in globals.yml like
>>> >> monitoring: grafana or
>>> >> monitoring: sensu
>>> >>
>>> >> Comments questions or concerns welcome.
>>> >>
>>> >> Regards
>>> >> -steve
>>> >>
>>> >> On 7/22/16, 3:42 PM, "Stephen Hindle" <shindle at llnw.com> wrote:
>>> >>
>>> >> >Don't forget mewalds implementation as well - we now have 2
>>> monitoring
>>> >> >options for kolla :-)
>>> >> >
>>> >> >On Fri, Jul 22, 2016 at 3:15 PM, Steven Dake (stdake) <
>>> stdake at cisco.com>
>>> >> >wrote:
>>> >> >> Hi folks,
>>> >> >>
>>> >> >> At the midcycle we decided to push off implementing Monitoring
>>> until
>>> >> >>post
>>> >> >> Newton.  The rationale for this decision was that the core review
>>> team
>>> >> >>has
>>> >> >> enough on their plates and nobody was super keen to implement any
>>> >> >>monitoring
>>> >> >> solution given our other priorities.
>>> >> >>
>>> >> >> Like all good things, communities produce new folks that want to
>>> do new
>>> >> >> things, and Sensu was proposed as Kolla's monitoring solution
>>> (atleast
>>> >> >>the
>>> >> >> first one).  A developer that has done some good work has shown up
>>> to
>>> >> >>do the
>>> >> >> job as well :)  I have heard good things about Sensu, minus the
>>> fact
>>> >> >>that it
>>> >> >> is implemented in Ruby and I fear it may end up causing our gate a
>>> lot
>>> >> >>of
>>> >> >> hassle.
>>> >> >>
>>> >> >> https://review.openstack.org/#/c/341861/
>>> >> >>
>>> >> >>
>>> >> >> Anyway I think we can work through the gate problem.
>>> >> >>
>>> >> >> Does anyone have any better suggestion?  I'd like to unblock Dave's
>>> >> >> work
>>> >> >> which is blocked on a ­2 pending a complete discussion of our
>>> >> >> monitoring
>>> >> >> solution.  Note we may end up implementing more than one down the
>>> road
>>> >> >> ­
>>> >> >> Sensu is just where the original interest was.
>>> >> >>
>>> >> >> Please provide feedback, even if you don't have a preference,
>>> whether
>>> >> >>your a
>>> >> >> core reviewer or not.
>>> >> >>
>>> >> >> My take is we can merge this work in non-prioirty order, and if it
>>> >> >>makes the
>>> >> >> end of the cycle fantastic ­ if not, we can release it in Ocatta.
>>> >> >>
>>> >> >> Regards
>>> >> >> -steve
>>> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> >> >>
>>> >>_________________________________________________________________________
>>> >> >>_
>>> >> >> OpenStack Development Mailing List (not for usage questions)
>>> >> >> Unsubscribe:
>>> >> >>OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> >> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> >> >>
>>> >> >
>>> >> >
>>> >> >
>>> >> >--
>>> >> >Stephen Hindle - Senior Systems Engineer
>>> >> >480.807.8189 480.807.8189
>>> >> >www.limelight.com Delivering Faster Better
>>> >> >
>>> >> >Join the conversation
>>> >> >
>>> >> >at Limelight Connect
>>> >> >
>>> >> >--
>>> >> >The information in this message may be confidential.  It is intended
>>> >> >solely
>>> >> >for
>>> >> >the addressee(s).  If you are not the intended recipient, any
>>> disclosure,
>>> >> >copying or distribution of the message, or any action or omission
>>> taken
>>> >> >by
>>> >> >you
>>> >> >in reliance on it, is prohibited and may be unlawful.  Please
>>> immediately
>>> >> >contact the sender if you have received this message in error.
>>> >> >
>>> >> >
>>> >>
>>> >> >
>>> >__________________________________________________________________________
>>> >> >OpenStack Development Mailing List (not for usage questions)
>>> >> >Unsubscribe:
>>> >> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> >> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> >>
>>> >>
>>> >>
>>> __________________________________________________________________________
>>> >> OpenStack Development Mailing List (not for usage questions)
>>> >> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> >>
>>> >>
>>> __________________________________________________________________________
>>> >> OpenStack Development Mailing List (not for usage questions)
>>> >> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> >
>>> >
>>> >
>>> >
>>> __________________________________________________________________________
>>> > OpenStack Development Mailing List (not for usage questions)
>>> > Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> >
>>>
>>>
>>>
>>> --
>>> Stephen Hindle - Senior Systems Engineer
>>> 480.807.8189 480.807.8189
>>> www.limelight.com Delivering Faster Better
>>>
>>> Join the conversation
>>>
>>> at Limelight Connect
>>>
>>> --
>>> The information in this message may be confidential.  It is intended
>>> solely
>>> for
>>> the addressee(s).  If you are not the intended recipient, any disclosure,
>>> copying or distribution of the message, or any action or omission taken
>>> by
>>> you
>>> in reliance on it, is prohibited and may be unlawful.  Please immediately
>>> contact the sender if you have received this message in error.
>>>
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>>
>> --
>> Mobil: +49 176 10567592
>> E-Mail: mewald at evoila.de
>>
>> evoila GmbH
>> Wilhelm-Theodor-Römheld-Str. 34
>> 55130 Mainz
>> Germany
>>
>> Geschäftsführer: Johannes Hiemer
>>
>> Amtsgericht Mainz HRB 42719
>>
>> Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte
>> Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail
>> irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und
>> vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte
>> Weitergabe dieser Mail ist nicht gestattet.
>>
>> This e-mail may contain confidential and/or privileged information. If
>> You are not the intended recipient (or have received this e-mail in error)
>> please notify the sender immediately and destroy this e-mail. Any
>> unauthorised copying, disclosure or distribution of the material in this
>> e-mail is strictly forbidden.
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Mobil: +49 176 10567592
E-Mail: mewald at evoila.de

evoila GmbH
Wilhelm-Theodor-Römheld-Str. 34
55130 Mainz
Germany

Geschäftsführer: Johannes Hiemer

Amtsgericht Mainz HRB 42719

Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte
Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail
irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und
vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte
Weitergabe dieser Mail ist nicht gestattet.

This e-mail may contain confidential and/or privileged information. If You
are not the intended recipient (or have received this e-mail in error)
please notify the sender immediately and destroy this e-mail. Any
unauthorised copying, disclosure or distribution of the material in this
e-mail is strictly forbidden.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160724/520ad8a7/attachment.html>