<br><br><div class="gmail_quote">On Wed, Oct 10, 2012 at 10:01 PM, Deok-June Yi <span dir="ltr"><<a href="mailto:june.yi@samsung.com" target="_blank">june.yi@samsung.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<p>Hi Eoghan,</p><div class="im">
<p>> 2. What technology is the metrics store based on, at a high level?<br>> (e.g. RDBMS accessed via sqlalchemy, noSQL store such as MongoDB,<br>> etc.)<br>><br>> 3. How are you performing datapoint aggregation? (e.g. eagerly on<br>
> metric ingestion, versus lazily on get)<br>></p>
</div><p>In addition to Joonwon's,</p>
<p>Synaps basically stores metric data aggregated with 1 minute resolution in hybrid way, in-memory of storm workers and Cassandra database. And it rolls those aggregated datapoints up again when another resolutions of statistics are requested. </p>
<p>Storm holds time-series datapoints within flexible sliding window per metric based on its longest period of alarm so that it can evaluate alarms without any reading operation from Cassandra database.</p>
<p>Synaps was designed with inspiration by following posts:</p>
<p>[1] <a href="https://www.cloudkick.com/blog/2010/mar/02/4_months_with_cassandra/" target="_blank">https://www.cloudkick.com/blog/2010/mar/02/4_months_with_cassandra/</a><br>[2] <a href="http://pyvideo.org/video/675/storm-the-hadoop-of-realtime-stream-processing" target="_blank">http://pyvideo.org/video/675/storm-the-hadoop-of-realtime-stream-processing</a></p>
<p><br>Hi Endre,</p><div class="im">
<p>> But how does Synaps then difffer from Ceilometer? It looks to be duplicated<br>> efforts here?<br>> </p>
</div><p>I think I don't understand Ceilometer enough but as I know its main purpose is to be a billing system based on hourly collected data while Synaps is focusing to be a monitoring service which deals with more amount of data more frequently.</p>
<p></p></div></blockquote><div><br></div><div>Ceilometer collects data for use by a billing system, but it is not itself a billing system. It sounds like Synaps collects metrics far more frequently than ceilometer does.</div>
<div><br></div><div>Can you share a list of the things Synaps measures?</div><div><br></div><div>Doug</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><p> </p>
<p>Thank you,<br>June Yi</p>
<p> </p>
<p> </p>
<p>------- <b>Original Message</b> -------</p>
<p><b>Sender</b> : Endre Karlson<<a href="mailto:endre.karlson@gmail.com" target="_blank">endre.karlson@gmail.com</a>></p>
<p><b>Date</b> : 2012-10-11 06:04 (GMT+09:00)</p><div><div class="h5">
<p><b>Title</b> : Re: [openstack-dev] Introducing Synaps project that provides AWS CloudWatch compatible API</p>
<p> </p>But how does Synaps then difffer from Ceilometer? It looks to be duplicated efforts here?
<div><br></div>
<div>Endre.<br><br>
<div class="gmail_quote">2012/10/10 Joonwon Lee <span dir="ltr"><<a href="mailto:joonwon7.lee@samsung.com" target="_blank">joonwon7.lee@samsung.com</a>></span><br>
<blockquote style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid" class="gmail_quote">Hi, Eoghan,<br>1. No, we don't have any relation with ceilometer.<br>
2. Cassandra and Storm (as explained in <a href="http://wiki.openstack.org/Synaps" target="_blank">http://wiki.openstack.org/Synaps</a>)<br>3. As far as I understand, aggregation is performed lazily on the request.<br> (Metrics are collected every minute and aggregated according to the request.)<br>
<br>Deok-June Yi can explain with more details tomorrow, if you want.<br>Thanks,<br>Joonwon Lee<br>
<div><br>------- Original Message -------<br>Sender : Eoghan Glynn<<a href="mailto:eglynn@redhat.com" target="_blank">eglynn@redhat.com</a>><br></div>Date : Oct 10, 2012 21:14 (GMT+09:00)<br>
<div>Title : Re: [openstack-dev] Introducing Synaps project that provides AWS CloudWatch compatible API<br><br><br></div>
<div>
<div>Hi Joonwon Lee,<br><br>> Unfortunately, we could not attend the Design Summit. We also need<br>> some time<br>> before opening our source codes, mostly for our internal process. We<br>> will try to shorten this time.<br>
<br>It's unfortunate you guys won't be at the summit, as it would have<br>been great to discuss in that forum.<br><br>But that said, the quicker you can push the code through your internal<br>process, the better.<br>
<br><br>> Now we're preparing the overview of our project and other<br>> documentation.<br>> If you have any questions, feel free to ask them at any time.<br><br>A few immediate questions that spring to mind:<br>
<br>1. Are you re-using any of the ceilometer[1] infrastructure for<br> your metrics collection pipeline?<br><br>2. What technology is the metrics store based on, at a high level?<br> (e.g. RDBMS accessed via sqlalchemy, noSQL store such as MongoDB,<br>
etc.)<br><br>3. How are you performing datapoint aggregation? (e.g. eagerly on<br> metric ingestion, versus lazily on get)<br><br>Cheers,<br>Eoghan<br><br>[1] <a href="http://ceilometer.readthedocs.org/en/latest/architecture.html" target="_blank">http://ceilometer.readthedocs.org/en/latest/architecture.html</a><br>
<br><br>> Here are answers of June Yi for Steven Hardy's questions below.<br>> > - Is the metric collection secure/authenticated?<br>> > - Are metrics collected from the hypervisors or in-instance?<br>> > - Can service monitoring (in instance) metrics be collected?<br>
> > - Is the API authenticated (ie via keystone ec2 credentials?)<br>><br>> All metrics are collected using the API. The API is authenticated via<br>> auth<br>> module came from Nova.Authenticate WSGI middleware which supports AWS<br>
> signature v2. It works well with LdapDriver. But we did not tested if<br>> it is<br>> working with keystone yet.<br>><br>> Default metrics (i.e. CPUUtilization, disk IO, network IO) are<br>> collected<br>
> from hypervisors. Custom metrics can be collected from in-instance or<br>> any other source.<br>><br>> Regards,<br>> --<br>> Joonwon Lee, principal software engineer, Samsung SDS<br>><br>><br>> ------- Original Message -------<br>
> Sender : Eoghan Glynn<br></div></div>
<div>
<div>> Date : 2012-10-10 18:59 (GMT+09:00)<br>> Title : Re: [openstack-dev] Introducing Synaps project that provides<br>> AWS CloudWatch compatible API<br>><br>><br>><br>> > > > I'm working for Synaps project that provides AWS CloudWatch<br>
> > > > compatible API at Samsung SDS.<br>> > > > Currently, we have plan to open the project to the community.<br>> > > ><br>> > > > Here I introduce Synaps project:<br>> > > > <a href="http://wiki.openstack.org/Synaps" target="_blank">http://wiki.openstack.org/Synaps</a><br>
> > ><br>> > > Details are a bit scarce but at first glance there seem to be an<br>> > > overlap<br>> > > with the CloudWatch work in the Heat project ?<br>> ><br>> > There certainly seems to be from the description, but pretty hard<br>
> > to<br>> > evaluate until we see some code ;)<br>><br>> Yes, I would second that.<br>><br>> Once the code is out the open, the community will be much better<br>> placed<br>> to judge which approach will make sense to concentrate efforts on.<br>
><br>> June Yi - are you planning on attending the design summit?<br>><br>> Cheers,<br>> Eoghan<br>><br>><br>> > > Any chance the two groups could meet at the Design Summit next<br>> > > week<br>
> > > and<br>> > > make sure we don't duplicate effort here ?<br>> > ><br>> > > Angus Saskeld has a session on CloudWatch in the Incubation room:<br>> > > <a href="http://openstacksummitfall2012.sched.org/event/60b0bcf1253efcb3dd72dff531d59d26" target="_blank">http://openstacksummitfall2012.sched.org/event/60b0bcf1253efcb3dd72dff531d59d26</a><br>
> ><br>> > Angus Salkeld and Steve Dake will be at the Summit and could<br>> > discuss,<br>> > also<br>> > possible overlap with metric collection in ceilometer (has been<br>> > discussed<br>
> > previously).<br>> ><br>> > I have several questions which will be make/break for the heat<br>> > usage<br>> > of<br>> > Cloudwatch:<br>> ><br>> > - Is the metric collection secure/authenticated?<br>
> > - Are metrics collected from the hypervisors or in-instance?<br>> > - Can service monitoring (in instance) metrics be collected?<br>> > - Is the API authenticated (ie via keystone ec2 credentials?)<br>
> ><br>> > Many more questions, need to see code and evaluate if we could use<br>> > this for<br>> > heat or if we continue with our own implementation.<br>> ><br>> > --<br>> > Steve Hardy<br>
> > Red Hat Engineering, Cloud<br>> ><br>> > _______________________________________________<br>> > OpenStack-dev mailing list<br>> > <a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
> > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>> ><br>><br>> _______________________________________________<br>
> OpenStack-dev mailing list<br>> <a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
_______________________________________________<br>OpenStack-dev mailing list<br><a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div><br></div>
<p> </p>
</div></div><table>
<tbody>
<tr>
<td>
<p><img border="0" width="520"></p></td></tr></tbody></table></div><img border="0" width="0" height="0"><br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br>