<div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Apr 30, 2012 at 3:43 PM, Loic Dachary <span dir="ltr"><<a href="mailto:loic@enovance.com" target="_blank">loic@enovance.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><div><div class="h5">
On 04/30/2012 08:03 PM, Doug Hellmann wrote:
<blockquote type="cite">
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Mon, Apr 30, 2012 at 11:43 AM, Loic
Dachary <span dir="ltr"><<a href="mailto:loic@enovance.com" target="_blank">loic@enovance.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>
<div> On 04/30/2012 03:49 PM, Doug Hellmann
wrote:
<blockquote type="cite">
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Mon, Apr 30, 2012 at
6:46 AM, Loic Dachary <span dir="ltr"><<a href="mailto:loic@enovance.com" target="_blank">loic@enovance.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>On 04/30/2012 12:15 PM, Loic Dachary
wrote:<br>
> We could start a discussion from the
content of the following sections:<br>
><br>
> <a href="http://wiki.openstack.org/EfficientMetering#Counters" target="_blank">http://wiki.openstack.org/EfficientMetering#Counters</a><br>
</div>
I think the rationale of the counter
aggregation needs to be explained. My
understanding is that the metering system will
be able to deliver the following information:
10 floating IPv4 addresses were allocated to
the tenant during three months and were leased
from provider NNN. From this, the billing
system could add a line to the invoice : 10
IPv4, $N each = $10xN because it has been
configured to invoice each IPv4 leased from
provider NNN for $N.<br>
<br>
It is not the purpose of the metering system
to display each IPv4 used, therefore it only
exposes the aggregated information. The
counters define how the information should be
aggregated. If the idea was to expose each
resource usage individually, defining counters
would be meaningless as they would duplicate
the activity log from each OpenStack
component.<br>
<br>
What do you think ?<br>
</blockquote>
<div><br>
</div>
<div>At DreamHost we are going to want to show
each individual resource (the IPv4 address,
the instance, etc.) along with the charge
information. Having the metering system
aggregate that data will make it
difficult/impossible to present the bill
summary and detail views that we want. It
would be much more useful for us if it tracked
the usage details for each resource, and let
us aggregate the data ourselves.</div>
<div><br>
</div>
<div>If other vendors want to show the data
differently, perhaps we should provide
separate APIs for retrieving the detailed and
aggregate data.</div>
<div><br>
</div>
<div>Doug</div>
<div><br>
</div>
</div>
</div>
</blockquote>
</div>
</div>
Hi,<br>
<br>
For the record, here is the unfinished conversation we had
on IRC<br>
<br>
<span style="font-weight:normal"><font><font color="#af7f00">(04:29:06 PM) </font></font></span><span style="font-weight:bold;color:#af7f00">dhellmann: </span>dachary,
did you see my reply about counter definitions on the list
today?<br>
<span style="font-weight:normal"><font><font color="#204a87">(04:39:05 PM) </font></font></span><span style="font-weight:bold;color:#204a87">dachary: </span>It
means some counters must not be aggregated. Only the
amount associated with it is but there is one counter per
IP. <br>
<span style="font-weight:normal"><font><font color="#204a87">(04:55:01 PM) </font></font></span><span style="font-weight:bold;color:#204a87">dachary: </span>dhellmann:
what about this :the id of the ressource controls the
agregation of all counters : if it is missing, all
resources of the same kind and their measures are
aggregated. Otherwise only the measures are agreggated. <a href="http://wiki.openstack.org/EfficientMetering?action=diff&rev2=40&rev1=39" target="_blank">http://wiki.openstack.org/EfficientMetering?action=diff&rev2=40&rev1=39</a><br>
<span style="font-weight:normal"><font><font color="#204a87">(04:55:58 PM) </font></font></span><span style="font-weight:bold;color:#204a87">dachary: </span>it
makes me a little unconfortable to define such an "ad-hoc"
grouping <br>
<span style="font-weight:normal"><font><font color="#204a87">(04:56:53 PM) </font></font></span><span style="font-weight:bold;color:#204a87">dachary: </span>i.e.
you actuall control the aggregation by chosing which value
to put in the id column<br>
<span style="font-weight:normal"><font><font color="#204a87">(04:58:43 PM) </font></font></span><span style="font-weight:bold;color:#204a87">dachary: </span>s/actuall/actually/<br>
<span style="font-weight:normal"><font><font color="#062585">(05:05:38 PM) </font></font></span><span style="font-weight:bold;color:#062585">***dachary </span>reading
<a href="http://www.ogf.org/documents/GFD.98.pdf" target="_blank">http://www.ogf.org/documents/GFD.98.pdf</a><br>
<span style="font-weight:normal"><font><font color="#204a87">(05:05:54 PM) </font></font></span><span style="font-weight:bold;color:#204a87">dachary: </span>I
feel like we're trying to resolve a non problem here<br>
<span style="font-weight:normal"><font><font color="#204a87">(05:08:42 PM) </font></font></span><span style="font-weight:bold;color:#204a87">dachary: </span>values
need to be aggregated. The raw input is a full description
of the resource and a value ( gauge ). The question is how
to control the aggregation in a reasonably flexible way. <br>
<span style="font-weight:normal"><font><font color="#204a87">(05:11:34 PM) </font></font></span><span style="font-weight:bold;color:#204a87">dachary: </span>The
definition of a counter could probably be described as :
the id of a resource and code to fill each column
associated with it.<br>
<br>
I tried to append the following, but the wiki kept
failing. <br>
<br>
Propose that the counters are defined by a function
instead of being fixed. That helps addressing the issue of
aggregating the bandwidth associated to a given IP into a
single counter.<br>
<br>
Alternate idea : <br>
* a counter is defined by<br>
* a name ( o1, n2, etc. ) that uniquely identifies the
nature of the measure ( outbound internet transit, amount
of RAM, etc. )<br>
* the component in which it can be found ( nova, swift
etc.)<br>
* and by columns, each one is set with the result of
aggregate(find(record),record) where<br>
* find() looks for the existing column as found by
selecting with the unique key ( maybe the name and the
resource id )<br>
* record is a detailed description of the metering event
to be aggregated ( <a href="http://wiki.openstack.org/SystemUsageData#compute.instance.exists:" target="_blank">http://wiki.openstack.org/SystemUsageData#compute.instance.exists:</a>
)<br>
* the aggregate() function returns the updated row. By
default it just += the counter value with the old row
returned by find()</div>
</blockquote>
<div><br>
</div>
<div>Would we want aggregation to occur within the database
where we are collecting events, or should that move
somewhere else?</div>
</div>
</div>
</blockquote></div></div>
I assume the events collected by the metering agents will all be
archived for auditing (or re-building the database)
<a href="http://wiki.openstack.org/EfficientMetering?action=diff&rev2=45&rev1=44" target="_blank">http://wiki.openstack.org/EfficientMetering?action=diff&rev2=45&rev1=44</a><br>
<br>
Therefore the aggregation should occur when the database is updated
to account for a new event.<br>
<br>
Does this make sense ? I may have misunderstood part of your
question.<br></div></blockquote><div><br></div><div>I guess what I don't understand is why the aggregated data is written back to the metering database at all. If it's in the same database, it seems like it should be in a different "table" (or equivalent) so the original data is left alone.</div>
<div><br></div><div>Maybe it's time to start focusing these discussions on user stories?</div><div><br></div></div></div>