Open Stack

Maybe it's related to this in the wiki:
http://wiki.openstack.org/ResourceMonitorAlertsandNotifications  and
related BP:
https://blueprints.launchpad.net/openstack-devops/+spec/resource-monitor-alerts-and-notifications

Not sure if anybody started working on that. Hope it helps.

On Thu, Oct 11, 2012 at 10:50 AM, Doug Hellmann <doug.hellmann at dreamhost.com
> wrote:

>
>
> On Wed, Oct 10, 2012 at 10:01 PM, Deok-June Yi <june.yi at samsung.com>wrote:
>
>>  Hi Eoghan,
>>
>> > 2. What technology is the metrics store based on, at a high level?
>> >    (e.g. RDBMS accessed via sqlalchemy, noSQL store such as MongoDB,
>> >     etc.)
>> >
>> > 3. How are you performing datapoint aggregation? (e.g. eagerly on
>> >    metric ingestion, versus lazily on get)
>> >
>>
>> In addition to Joonwon's,
>>
>> Synaps basically stores metric data aggregated with 1 minute resolution
>> in hybrid way, in-memory of storm workers and Cassandra database. And it
>> rolls those aggregated datapoints up again when another resolutions of
>> statistics are requested.
>>
>> Storm holds time-series datapoints within flexible sliding window per
>> metric based on its longest period of alarm so that it can evaluate alarms
>> without any reading operation from Cassandra database.
>>
>> Synaps was designed with inspiration by following posts:
>>
>> [1] https://www.cloudkick.com/blog/2010/mar/02/4_months_with_cassandra/
>> [2]
>> http://pyvideo.org/video/675/storm-the-hadoop-of-realtime-stream-processing
>>
>>
>> Hi Endre,
>>
>> > But how does Synaps then difffer from Ceilometer? It looks to be
>> duplicated
>> > efforts here?
>> >
>>
>> I think I don't understand Ceilometer enough but as I know its main
>> purpose is to be a billing system based on hourly collected data while
>> Synaps is focusing to be a monitoring service which deals with more amount
>> of data more frequently.
>>
>>
> Ceilometer collects data for use by a billing system, but it is not itself
> a billing system. It sounds like Synaps collects metrics far more
> frequently than ceilometer does.
>
> Can you share a list of the things Synaps measures?
>
> Doug
>
>
>>
>>
>> Thank you,
>> June Yi
>>
>>
>>
>>
>>
>> ------- *Original Message* -------
>>
>> *Sender* : Endre Karlson<endre.karlson at gmail.com>
>>
>> *Date* : 2012-10-11 06:04 (GMT+09:00)
>>
>> *Title* : Re: [openstack-dev] Introducing Synaps project that provides
>> AWS CloudWatch compatible API
>>
>>
>> But how does Synaps then difffer from Ceilometer? It looks to be
>> duplicated efforts here?
>>
>> Endre.
>>
>> 2012/10/10 Joonwon Lee <joonwon7.lee at samsung.com>
>>
>>> Hi, Eoghan,
>>> 1. No, we don't have any relation with ceilometer.
>>> 2. Cassandra and Storm (as explained in http://wiki.openstack.org/Synaps
>>> )
>>> 3. As far as I understand, aggregation is performed lazily on the
>>> request.
>>>  (Metrics are collected every minute and aggregated according to the
>>> request.)
>>>
>>> Deok-June Yi can explain with more details tomorrow, if you want.
>>> Thanks,
>>> Joonwon Lee
>>>
>>> ------- Original Message -------
>>> Sender : Eoghan Glynn<eglynn at redhat.com>
>>> Date : Oct 10, 2012 21:14 (GMT+09:00)
>>> Title : Re: [openstack-dev] Introducing Synaps project that provides AWS
>>> CloudWatch compatible API
>>>
>>>
>>>  Hi Joonwon Lee,
>>>
>>> > Unfortunately, we could not attend the Design Summit. We also need
>>> > some time
>>> > before opening our source codes, mostly for our internal process. We
>>> > will try to shorten this time.
>>>
>>> It's unfortunate you guys won't be at the summit, as it would have
>>> been great to discuss in that forum.
>>>
>>> But that said, the quicker you can push the code through your internal
>>> process, the better.
>>>
>>>
>>> > Now we're preparing the overview of our project and other
>>> > documentation.
>>> > If you have any questions, feel free to ask them at any time.
>>>
>>> A few immediate questions that spring to mind:
>>>
>>> 1. Are you re-using any of the ceilometer[1] infrastructure for
>>>    your metrics collection pipeline?
>>>
>>> 2. What technology is the metrics store based on, at a high level?
>>>    (e.g. RDBMS accessed via sqlalchemy, noSQL store such as MongoDB,
>>>     etc.)
>>>
>>> 3. How are you performing datapoint aggregation? (e.g. eagerly on
>>>    metric ingestion, versus lazily on get)
>>>
>>> Cheers,
>>> Eoghan
>>>
>>> [1] http://ceilometer.readthedocs.org/en/latest/architecture.html
>>>
>>>
>>> > Here are answers of June Yi for Steven Hardy's questions below.
>>> > > - Is the metric collection secure/authenticated?
>>> > > - Are metrics collected from the hypervisors or in-instance?
>>> > > - Can service monitoring (in instance) metrics be collected?
>>> > > - Is the API authenticated (ie via keystone ec2 credentials?)
>>> >
>>> > All metrics are collected using the API. The API is authenticated via
>>> > auth
>>> > module came from Nova.Authenticate WSGI middleware which supports AWS
>>> > signature v2. It works well with LdapDriver. But we did not tested if
>>> > it is
>>> > working with keystone yet.
>>> >
>>> > Default metrics (i.e. CPUUtilization, disk IO, network IO) are
>>> > collected
>>> > from hypervisors. Custom metrics can be collected from in-instance or
>>> > any other source.
>>> >
>>> > Regards,
>>> > --
>>> > Joonwon Lee, principal software engineer, Samsung SDS
>>> >
>>> >
>>> > ------- Original Message -------
>>> > Sender : Eoghan Glynn
>>>  > Date : 2012-10-10 18:59 (GMT+09:00)
>>> > Title : Re: [openstack-dev] Introducing Synaps project that provides
>>> > AWS CloudWatch compatible API
>>> >
>>> >
>>> >
>>> > > > > I'm working for Synaps project that provides AWS CloudWatch
>>> > > > > compatible API at Samsung SDS.
>>> > > > > Currently, we have plan to open the project to the community.
>>> > > > >
>>> > > > > Here I introduce Synaps project:
>>> > > > > http://wiki.openstack.org/Synaps
>>> > > >
>>> > > > Details are a bit scarce but at first glance there seem to be an
>>> > > > overlap
>>> > > > with the CloudWatch work in the Heat project ?
>>> > >
>>> > > There certainly seems to be from the description, but pretty hard
>>> > > to
>>> > > evaluate until we see some code ;)
>>> >
>>> > Yes, I would second that.
>>> >
>>> > Once the code is out the open, the community will be much better
>>> > placed
>>> > to judge which approach will make sense to concentrate efforts on.
>>> >
>>> > June Yi - are you planning on attending the design summit?
>>> >
>>> > Cheers,
>>> > Eoghan
>>> >
>>> >
>>> > > > Any chance the two groups could meet at the Design Summit next
>>> > > > week
>>> > > > and
>>> > > > make sure we don't duplicate effort here ?
>>> > > >
>>> > > > Angus Saskeld has a session on CloudWatch in the Incubation room:
>>> > > >
>>> http://openstacksummitfall2012.sched.org/event/60b0bcf1253efcb3dd72dff531d59d26
>>> > >
>>> > > Angus Salkeld and Steve Dake will be at the Summit and could
>>> > > discuss,
>>> > > also
>>> > > possible overlap with metric collection in ceilometer (has been
>>> > > discussed
>>> > > previously).
>>> > >
>>> > > I have several questions which will be make/break for the heat
>>> > > usage
>>> > > of
>>> > > Cloudwatch:
>>> > >
>>> > > - Is the metric collection secure/authenticated?
>>> > > - Are metrics collected from the hypervisors or in-instance?
>>> > > - Can service monitoring (in instance) metrics be collected?
>>> > > - Is the API authenticated (ie via keystone ec2 credentials?)
>>> > >
>>> > > Many more questions, need to see code and evaluate if we could use
>>> > > this for
>>> > > heat or if we continue with our own implementation.
>>> > >
>>> > > --
>>> > > Steve Hardy
>>> > > Red Hat Engineering, Cloud
>>> > >
>>> > > _______________________________________________
>>> > > OpenStack-dev mailing list
>>> > > OpenStack-dev at lists.openstack.org
>>> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> > >
>>> >
>>> > _______________________________________________
>>> > OpenStack-dev mailing list
>>> > OpenStack-dev at lists.openstack.org
>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20121011/7c772430/attachment.html>