<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
On 10/03/2012 05:13 PM, Doug Hellmann wrote:
<blockquote
cite="mid:CADb+p3RBgov8CsFEiWRJm9O8BN7xV7FRW774VWWuTQGTzmOpaw@mail.gmail.com"
type="cite"><br>
<br>
<div class="gmail_quote">On Wed, Oct 3, 2012 at 10:10 AM, Gary
Kotton <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:gkotton@redhat.com" target="_blank">gkotton@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div class="im"> On 10/03/2012 02:42 PM, Doug Hellmann
wrote:
<blockquote type="cite">On Wed, Oct 3, 2012 at 8:30 AM,
Gary Kotton <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:gkotton@redhat.com" target="_blank">gkotton@redhat.com</a>></span>
wrote:</blockquote>
</div>
<div class="im">
<blockquote type="cite">
<div>
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi,<br>
I wanted to ask a few additional questions
regarding the patch - <a moz-do-not-send="true"
href="https://review.openstack.org/#/c/13943/"
target="_blank">https://review.openstack.org/#/c/13943/</a>
- aka quantum-usage-audit:-<br>
1. Do you guys intend to take care of the router
and floating IP support? This seems to be lacking
at the moment. I am not sure if this is supported
by notifications at the moment.<br>
</blockquote>
<div><br>
</div>
<div>We are currently polling for floating IP
status. Julien did that work, IIRC, and I think
polling was used because there were no
notifications. We should be able to contribute
some help to add notifications during grizzly,
which would let us eliminate the polling on our
side.</div>
</div>
</div>
</blockquote>
<br>
</div>
OK. This is certainly something that is missing in Quantum.
I think that we should open a bug for tracking.</div>
</blockquote>
<div><br>
</div>
<div>I agree. If you mark the bug as affecting ceilometer, too,
we'll be able to coordinate the work between the projects.</div>
</div>
</blockquote>
<br>
Done. <a class="moz-txt-link-freetext" href="https://bugs.launchpad.net/quantum/+bug/1060985">https://bugs.launchpad.net/quantum/+bug/1060985</a><br>
<blockquote
cite="mid:CADb+p3RBgov8CsFEiWRJm9O8BN7xV7FRW774VWWuTQGTzmOpaw@mail.gmail.com"
type="cite">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div class="im"><br>
<blockquote type="cite">
<div>
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
2. I think that the reporting can be optimized.
Can you please elaborate a little more on how you
see things. I would hope that this is going to be
something that is going to be discussed at the
summit. Did you guys consider using a pull from
Quantum using the API - similar to the L3 agent?<br>
</blockquote>
<div><br>
</div>
<div>We do have some polling, although I think it's
going straight to the database right now (fixing
that is another goal, all of this is a work in
progress).</div>
</div>
</div>
</blockquote>
<br>
</div>
Is the polling via the quantum client?</div>
</blockquote>
<div><br>
</div>
<div>No, it's going directly to the database. See <a
moz-do-not-send="true"
href="https://github.com/stackforge/ceilometer/blob/master/ceilometer/network/floatingip.py">https://github.com/stackforge/ceilometer/blob/master/ceilometer/network/floatingip.py</a></div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div class="im"><br>
<blockquote type="cite">
<div>
<div class="gmail_quote">
<div><br>
</div>
<div>I'm not sure what you mean by "optimized"
though. Having a standalone app generate data over
the message bus seems like it would have less
impact on quantum than if we were hitting the API
every few minutes. Perhaps there's some aspect to
the way the data is gathered that I don't
understand though. The ultimate goal is to have
quantum emitting the audit notifications on its
own in whatever way combines efficiency and
accuracy. In nova and cinder the audit messages
are disabled by default, so a deployer has to set
a configuration option to enable them (for nova)
or enable a cron job (for cinder).</div>
</div>
</div>
</blockquote>
<br>
</div>
I was originally thinking of having a timestamp and then
doing the updates to ceilometer would be done according to
the timestamp. Say for example having a "Last Updated" field
could ensure that there is less data sent every interval. At
the moment there will be a network burst every time the
audit is run.</div>
</blockquote>
<div><br>
</div>
<div>Ah, I see. There are two reasons we don't want a timestamp
or other criteria to throttle those updates. First, we don't
want to manage state between the two systems (or potentially
more systems if other clients start listening for the
notifications). Quantum shouldn't have to worry about what
ceilometer may or may not already know to try to decide what
to tell us about next.</div>
<div><br>
</div>
<div>Most importantly, though, the entire design for ceilometer
depends on receiving regular updates. We don't treat "create"
and "delete" events in any special way. Every incoming event
is just an update to the state of a resource. The first event
we see for a resource/meter combination causes the new
resource to be created in our database, but that event doesn't
have to be a "create" event. Treating all events in the same
way simplifies the message handling logic in the notification
handlers, the API implementation, and client-side use of the
API. The API lets the caller ask questions about resource
usage in a lot of different ways, since different operators
will bill for different usage, at different intervals, using
different criteria (possibly even using metadata settings for
the resource itself, with instance flavor being the most
obvious example but the location of the instance may be
another factor). Because we don't know exactly what the users
of the API are going to ask for, we cannot aggregate the data
significantly. If we depended on special handling for
different notification event types, and we miss an event, a
resource may be left off of a customer's bill. But by using
regular updates, and not treating any of the events as special
triggers, we're able to be more flexible and more robust at
the same time.</div>
<div><br>
</div>
<div>If there is going to be too much data to update ceilometer
at at once, we could send the updates in batches. However,
quantum should never assume that ceilometer (or any other
listener) knows anything at all, so it should eventually (in a
relatively small amount of time) iterate over all of its
resources and send audit events.</div>
</div>
</blockquote>
<br>
True. I am not really sure why the 'exists' need to be sent every
interval when create and delete are sent when the actual event takes
place. If messages can get lost then there seems to be a problem
with the model. Hopefully we can talk about this in more detail at
summit.<br>
<br>
<blockquote
cite="mid:CADb+p3RBgov8CsFEiWRJm9O8BN7xV7FRW774VWWuTQGTzmOpaw@mail.gmail.com"
type="cite">
<div class="gmail_quote">
<div><br>
</div>
<div>One thing we could do is split up the existing audit job
into separate scripts for each resource type. That way if a
deployer doesn't need to meter ports, for example, they could
avoid doing that processing.</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div class="im"><br>
<blockquote type="cite">
<div>
<div class="gmail_quote">
<div><br>
</div>
<div>We do have several summit sessions planned.
Ceilometer has a mini-track Monday morning with 3
sessions (<a moz-do-not-send="true"
href="http://wiki.openstack.org/EfficientMetering/GrizzlySummit"
target="_blank">http://wiki.openstack.org/EfficientMetering/GrizzlySummit</a>),
and at Dan's request I proposed a session for the
Quantum track specifically to talk about
integrating Ceilometer and Quantum.</div>
</div>
</div>
</blockquote>
<br>
</div>
Thanks
<div class="im"><br>
<blockquote type="cite">
<div>
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
3. Is this something that will be cherry picked to
stable quantum and in turn should be packaged? If
so I understand that this is a cron job that
should be run every X minutes?<br>
</blockquote>
<div> </div>
<div>That is the goal. It would be up to deployers
to set up the cron job, and we would need to
document that in the ceilometer documentation
(probably the quantum docs, too, but we'll
definitely have it in our docs).</div>
<div><br>
</div>
<div>Doug</div>
<div><br>
</div>
</div>
</div>
</blockquote>
<br>
</div>
</div>
</blockquote>
</div>
<br>
</blockquote>
<br>
</body>
</html>