Open Stack

Hello Angus,

Thanks a lot for considering Ceilometer for this very needed tool, but I
am afraid you are confusing what Ceilometer is and what it can and
cannot do.

As defined on the blueprint [1] and further refined on one of the Faq
answers on the same page, Ceilometer is a metering tool built for the
purpose of reporting information to rating and billing engines.  Even
though performance statistics requirements may look similar at first
sight, they really aren't and one tool cannot replace the other.

Here are a few example:

- our polling interval are 5 min or more.  Would this be acceptable for
a resource to remain unavailable that long?  Increasing this interval
would drastically increase our data volume, and since we need to keep
this data, volume does count.  There is a tool that we have to estimate
volumes [2]. I don't think your tool would need to keep as much
information for as long.

- Ceilometer gather data from the infrastructure and has no plan to help
monitor instances.  Wouldn't this be a requirement for CloudWatch?

- Ceilometer has strong requirements for traceability of information,
something that you would not care about to collect performance metrics.

- Ceilometer uses its own message queue to ensure proper/lossless
transmission of information while a stats tool could happily use UDP
which would be much more efficient transport given the lighter
requirements.

Sorry to push you back on this, but I am afraid that the requirements
you expose would not match our own set of requirements.  I do value your
objective highly however, and hope that such a tool would be made
available for OpenStack deployments, as I can see many uses cases this
would solve.

We still have a topic about Heat integration with Ceilometer, as it make
full sense to have metering information from Heat too.  Dave is welcome
to join us tomorrow and discuss further.

Best,
Nick

[1] http://wiki.openstack.org/EfficientMetering
[2] http://wiki.openstack.org/EfficientMetering#Volume_of_data

On 08/08/2012 06:09 AM, Angus Salkeld wrote:
> Hi ceilometer team,
> 
> I can't make the meeting (it's 2am for me). But thought I could
> put some ideas down in email.
> 
> At the moment Heat has some CloudWatch'like functionality to achieve
> it's HA and autoscaling.
> I think this should be in ceilometer, not necessary the AWS api but
> something roughly equivalent.
> 
> What can you do with the CloudWatch api?
> ----------------------------------------
> - enable/disable what ever metrics are available on your resources
>   (minimum of 60 sec sample periods)
> - retrieve them (for up to 7 days after they were generated)
> - create/delete alarms based on your metrics (like alarm if metricX > 50)
> - view alarm history
> - generate data for custom metrics (by posting to the rest api)
> - change the state of the alarm manually (for debugging)
> - send alarms to either a autoscaling or messaging queue.
> 
> What can you achieve with CloudWatch?
> -------------------------------------
> - autoscaling: alarm on high/low cpu/disk/mem/<whatever> and trigger
>   new instances to be created/destroyed/resized.
> - service monitoring and HA: send data to custom metrics about service
>   health (httpd/mysqld/...) and act on it automatically.
> - extensible (by the end user) statistics gathering. So using the custom
>   metrics simply monitor something special on your instance.
> - Be quickly alerted of important events on your instances
>   (via the messaging queue) - we haven't done anything with this in Heat.
> - diagnostics: I think statistics is an important part of diagnostics both
>   to the user and administrator.
> 
> Why put it in ceilometer?
> -------------------------
> - expertise: put code of a common topic in one place where experts
>   in that area can congregate (ceilometer == all things statistics)
> - code reuse: I think there is a large common ground here
>   (make use of the pollsters to get metering and general statistics).
> 
> Some potential issues:
> ----------------------
> - increased volume of gathered statistics data (1 hourly -> 60 sec).
> [any more ...] 
> 
> I could do a talk about it at summit if need be, but it's worth
> throwing ideas about it now I think.
> 
> Note: Steven Dake will be at the ceilometer meeting to anwser any
> questions about Heat or CloudWatch.
> 
> 
> References:
> http://docs.amazonwebservices.com/AmazonCloudWatch/latest/DeveloperGuide/Welcome.html
> 
> http://docs.amazonwebservices.com/AmazonCloudWatch/latest/APIReference/Welcome.html
> 
> 
> Not sure if rackspace is going to (or has) opensource it's monitoring
> solution? Anyone?
> http://www.rackspace.com/cloud/public/monitoring/howitworks/
> 
> 
> Regards
> Angus Salkeld
> 
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 900 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20120808/7b761df7/attachment.pgp>