I posted some of this information on the changeset in gerrit, but I'm reproducing it here for the mailing list in case not everyone clicked through...<br><br><div class="gmail_quote">On Mon, Nov 5, 2012 at 12:17 PM, Monsyne Dragon <span dir="ltr"><<a href="mailto:mdragon@rackspace.com" target="_blank">mdragon@rackspace.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Er, I'm somewhat confused at this.<br>
<br>
Having the audit run from the volume manager as a periodic task is good (Nova already does this), but, first, running it every 60 seconds would produce *way* too much notification traffic,<br>
and second removing the audit_period breaks the usage notification model.<br></blockquote><div><br></div><div>Yes, I think we agree that's bad. We collect similar data every 10 minutes by default elsewhere, so we should make this change work the same way (I think that's what Julien's comments about the ticks_between_runs was about). Ultimately we want it to be configurable.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Cinder and Nova already produce immediate notifications (volume.create.* and volume.delete.* for Cinder, and many many types (compute.instance.create.*, etc) for Nova)<br></blockquote><div><br></div><div><span style="font-family:'Arial Unicode MS',Arial,sans-serif;background-color:rgb(255,255,255)">Those events are insufficient for the way ceilometer meters usage because we want to ensure we know about resources even if we miss their creation event, and we want to ensure we know about the metadata for a resource and how it changes over time.</span></div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
The audit events are meant for to provide a baseline for each period, so at any instant, you can know "the last period, there was these x volumes/instances, and I just got a create event, so now there is x+1",<br>
and you don't have to look back through history to the beginning of time to get an accurate count.<br>
<br>
I do not think this is necessary.<br></blockquote><div><br></div><div>Ceilometer works differently than the existing auditing system(s) because it is doing more than just auditing. <span style="background-color:rgb(255,255,255);font-family:'Arial Unicode MS',Arial,sans-serif">In order to support all of the queries needed for our use cases, we need the events to come more frequently than once every hour. The receiver will deal with auditing periods, aggregating data, etc. The sender should not have to worry about any of those requirements, so it becomes simpler and always sends regular exists notifications for each resource it knows about.</span></div>
<div><span style="background-color:rgb(255,255,255);font-family:'Arial Unicode MS',Arial,sans-serif"><br></span></div><div><span style="background-color:rgb(255,255,255);font-family:'Arial Unicode MS',Arial,sans-serif">In the changeset comments you mentioned that this would break existing tools looking for the "exists" events, and that's a good point. It looks like you and Julien are proposing using "volume.usage" instead of "volume.exists". I'm not sure that's quite the right description of the purpose of these events, but I don't have a better suggestion, so I can live with it. :-)</span></div>
<div><span style="background-color:rgb(255,255,255);font-family:'Arial Unicode MS',Arial,sans-serif"><br></span></div><div><span style="background-color:rgb(255,255,255);font-family:'Arial Unicode MS',Arial,sans-serif">Doug</span></div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="HOEnZb"><div class="h5"><br>
<br>
On Nov 5, 2012, at 9:34 AM, Julien Danjou wrote:<br>
<br>
> Hi,<br>
><br>
> I've been working to change the actual audit code used into Cinder (but<br>
> also in Nova) to a real-time one, so it's more useful to projects like<br>
> Ceilometer.<br>
><br>
> The implementation for Cinder is under review at:<br>
><br>
> <a href="https://review.openstack.org/#/c/15115/" target="_blank">https://review.openstack.org/#/c/15115/</a><br>
><br>
> Once this patch got accepted, I intend to port it to Nova in the same<br>
> fashion.<br>
><br>
> Now, Huang Zhiteng raises an issue that might also interest Nova team,<br>
> so I'm bringing this to this list.<br>
><br>
> My patch plugs the audit in a periodic task under the cinder-volume<br>
> daemon, since this is the one making more sense for cinder. For Nova,<br>
> that would probably be nova-compute.<br>
> The concern is that sending exists notifications might take too long and<br>
> blocks the daemon for too much time.<br>
><br>
> From my point of view, since each host is only sending notifications<br>
> about volumes (in Nova that would be instances) it handles, this is not<br>
> going to be a lot of them, and that shouldn't take the daemon too long<br>
> nor block it for a while. Compared to the burden of shipping another<br>
> daemon.<br>
><br>
> Also, while I'm at it, suggestions about a default auditing rate might<br>
> be welcome, since the default periodic_interval (60s) might be a little<br>
> too low. Maybe using ticks_between_runs would be a good idea?<br>
><br>
> --<br>
> Julien Danjou<br>
> # Free Software hacker & freelance<br>
> # <a href="http://julien.danjou.info" target="_blank">http://julien.danjou.info</a><br>
</div></div><div class="HOEnZb"><div class="h5">> _______________________________________________<br>
> OpenStack-dev mailing list<br>
> <a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div><br>