<div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Apr 30, 2012 at 11:43 AM, Loic Dachary <span dir="ltr"><<a href="mailto:loic@enovance.com" target="_blank">loic@enovance.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><div><div class="h5">
On 04/30/2012 03:49 PM, Doug Hellmann wrote:
<blockquote type="cite">
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Mon, Apr 30, 2012 at 6:46 AM, Loic
Dachary <span dir="ltr"><<a href="mailto:loic@enovance.com" target="_blank">loic@enovance.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>On 04/30/2012 12:15 PM, Loic Dachary wrote:<br>
> We could start a discussion from the content of the
following sections:<br>
><br>
> <a href="http://wiki.openstack.org/EfficientMetering#Counters" target="_blank">http://wiki.openstack.org/EfficientMetering#Counters</a><br>
</div>
I think the rationale of the counter aggregation needs to be
explained. My understanding is that the metering system will
be able to deliver the following information: 10 floating
IPv4 addresses were allocated to the tenant during three
months and were leased from provider NNN. From this, the
billing system could add a line to the invoice : 10 IPv4, $N
each = $10xN because it has been configured to invoice each
IPv4 leased from provider NNN for $N.<br>
<br>
It is not the purpose of the metering system to display each
IPv4 used, therefore it only exposes the aggregated
information. The counters define how the information should
be aggregated. If the idea was to expose each resource usage
individually, defining counters would be meaningless as they
would duplicate the activity log from each OpenStack
component.<br>
<br>
What do you think ?<br>
</blockquote>
<div><br>
</div>
<div>At DreamHost we are going to want to show each individual
resource (the IPv4 address, the instance, etc.) along with
the charge information. Having the metering system aggregate
that data will make it difficult/impossible to present the
bill summary and detail views that we want. It would be much
more useful for us if it tracked the usage details for each
resource, and let us aggregate the data ourselves.</div>
<div><br>
</div>
<div>If other vendors want to show the data differently,
perhaps we should provide separate APIs for retrieving the
detailed and aggregate data.</div>
<div><br>
</div>
<div>Doug</div>
<div><br>
</div>
</div>
</div>
</blockquote></div></div>
Hi,<br>
<br>
For the record, here is the unfinished conversation we had on IRC<br>
<br>
<span style="font-weight:normal"><font><font color="#af7f00">(04:29:06 PM) </font></font></span><span style="font-weight:bold;color:#af7f00">dhellmann: </span>dachary,
did you see my reply about counter definitions on the list today?<br>
<span style="font-weight:normal"><font><font color="#204a87">(04:39:05 PM) </font></font></span><span style="font-weight:bold;color:#204a87">dachary: </span>It
means some counters must not be aggregated. Only the amount
associated with it is but there is one counter per IP. <br>
<span style="font-weight:normal"><font><font color="#204a87">(04:55:01 PM) </font></font></span><span style="font-weight:bold;color:#204a87">dachary: </span>dhellmann:
what about this :the id of the ressource controls the agregation of
all counters : if it is missing, all resources of the same kind and
their measures are aggregated. Otherwise only the measures are
agreggated. <a href="http://wiki.openstack.org/EfficientMetering?action=diff&rev2=40&rev1=39" target="_blank">http://wiki.openstack.org/EfficientMetering?action=diff&rev2=40&rev1=39</a><br>
<span style="font-weight:normal"><font><font color="#204a87">(04:55:58 PM) </font></font></span><span style="font-weight:bold;color:#204a87">dachary: </span>it
makes me a little unconfortable to define such an "ad-hoc" grouping
<br>
<span style="font-weight:normal"><font><font color="#204a87">(04:56:53 PM) </font></font></span><span style="font-weight:bold;color:#204a87">dachary: </span>i.e.
you actuall control the aggregation by chosing which value to put in
the id column<br>
<span style="font-weight:normal"><font><font color="#204a87">(04:58:43 PM) </font></font></span><span style="font-weight:bold;color:#204a87">dachary: </span>s/actuall/actually/<br>
<span style="font-weight:normal"><font><font color="#062585">(05:05:38 PM) </font></font></span><span style="font-weight:bold;color:#062585">***dachary </span>reading
<a href="http://www.ogf.org/documents/GFD.98.pdf" target="_blank">http://www.ogf.org/documents/GFD.98.pdf</a><br>
<span style="font-weight:normal"><font><font color="#204a87">(05:05:54 PM) </font></font></span><span style="font-weight:bold;color:#204a87">dachary: </span>I feel
like we're trying to resolve a non problem here<br>
<span style="font-weight:normal"><font><font color="#204a87">(05:08:42 PM) </font></font></span><span style="font-weight:bold;color:#204a87">dachary: </span>values
need to be aggregated. The raw input is a full description of the
resource and a value ( gauge ). The question is how to control the
aggregation in a reasonably flexible way. <br>
<span style="font-weight:normal"><font><font color="#204a87">(05:11:34 PM) </font></font></span><span style="font-weight:bold;color:#204a87">dachary: </span>The
definition of a counter could probably be described as : the id of a
resource and code to fill each column associated with it.<br>
<br>
I tried to append the following, but the wiki kept failing. <br>
<br>
Propose that the counters are defined by a function instead of being
fixed. That helps addressing the issue of aggregating the bandwidth
associated to a given IP into a single counter.<br>
<br>
Alternate idea : <br>
* a counter is defined by<br>
* a name ( o1, n2, etc. ) that uniquely identifies the nature of
the measure ( outbound internet transit, amount of RAM, etc. )<br>
* the component in which it can be found ( nova, swift etc.)<br>
* and by columns, each one is set with the result of
aggregate(find(record),record) where<br>
* find() looks for the existing column as found by selecting with
the unique key ( maybe the name and the resource id )<br>
* record is a detailed description of the metering event to be
aggregated (
<a href="http://wiki.openstack.org/SystemUsageData#compute.instance.exists:" target="_blank">http://wiki.openstack.org/SystemUsageData#compute.instance.exists:</a> )<br>
* the aggregate() function returns the updated row. By default it
just += the counter value with the old row returned by find()</div></blockquote><div><br></div><div>Would we want aggregation to occur within the database where we are collecting events, or should that move somewhere else?</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000"><div class="im"><br>
<br>
Cheers<br>
<pre cols="3000">--
Loïc Dachary Chief Research Officer
// eNovance labs <a href="http://labs.enovance.com" target="_blank">http://labs.enovance.com</a>
// ✉ <a href="mailto:loic@enovance.com" target="_blank">loic@enovance.com</a> ☎ <a href="tel:%2B33%201%2049%2070%2099%2082" value="+33149709982" target="_blank">+33 1 49 70 99 82</a>
</pre>
</div></div>
</blockquote></div><br></div>