<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
On 04/30/2012 11:39 PM, Doug Hellmann wrote:
<blockquote
cite="mid:CADb+p3ScVxoddUnv4qwkc1ePkjC-o7xRzqZ_RzQO=XAcJDi+MQ@mail.gmail.com"
type="cite">
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Mon, Apr 30, 2012 at 3:43 PM, Loic
Dachary <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:loic@enovance.com" target="_blank">loic@enovance.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>
<div class="h5"> On 04/30/2012 08:03 PM, Doug Hellmann
wrote:
<blockquote type="cite">
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Mon, Apr 30, 2012 at
11:43 AM, Loic Dachary <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:loic@enovance.com"
target="_blank">loic@enovance.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0
0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>
<div> On 04/30/2012 03:49 PM, Doug
Hellmann wrote:
<blockquote type="cite">
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Mon, Apr
30, 2012 at 6:46 AM, Loic Dachary
<span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:loic@enovance.com"
target="_blank">loic@enovance.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div>On 04/30/2012 12:15 PM,
Loic Dachary wrote:<br>
> We could start a
discussion from the content of
the following sections:<br>
><br>
> <a
moz-do-not-send="true"
href="http://wiki.openstack.org/EfficientMetering#Counters"
target="_blank">http://wiki.openstack.org/EfficientMetering#Counters</a><br>
</div>
I think the rationale of the
counter aggregation needs to be
explained. My understanding is
that the metering system will be
able to deliver the following
information: 10 floating IPv4
addresses were allocated to the
tenant during three months and
were leased from provider NNN.
From this, the billing system
could add a line to the invoice
: 10 IPv4, $N each = $10xN
because it has been configured
to invoice each IPv4 leased from
provider NNN for $N.<br>
<br>
It is not the purpose of the
metering system to display each
IPv4 used, therefore it only
exposes the aggregated
information. The counters define
how the information should be
aggregated. If the idea was to
expose each resource usage
individually, defining counters
would be meaningless as they
would duplicate the activity log
from each OpenStack component.<br>
<br>
What do you think ?<br>
</blockquote>
<div><br>
</div>
<div>At DreamHost we are going to
want to show each individual
resource (the IPv4 address, the
instance, etc.) along with the
charge information. Having the
metering system aggregate that
data will make it
difficult/impossible to present
the bill summary and detail
views that we want. It would be
much more useful for us if it
tracked the usage details for
each resource, and let us
aggregate the data ourselves.</div>
<div><br>
</div>
<div>If other vendors want to show
the data differently, perhaps we
should provide separate APIs for
retrieving the detailed and
aggregate data.</div>
<div><br>
</div>
<div>Doug</div>
<div><br>
</div>
</div>
</div>
</blockquote>
</div>
</div>
Hi,<br>
<br>
For the record, here is the unfinished
conversation we had on IRC<br>
<br>
<span style="font-weight:normal"><font><font
color="#af7f00">(04:29:06 PM) </font></font></span><span
style="font-weight:bold;color:#af7f00">dhellmann:
</span>dachary, did you see my reply about
counter definitions on the list today?<br>
<span style="font-weight:normal"><font><font
color="#204a87">(04:39:05 PM) </font></font></span><span
style="font-weight:bold;color:#204a87">dachary:
</span>It means some counters must not be
aggregated. Only the amount associated with
it is but there is one counter per IP. <br>
<span style="font-weight:normal"><font><font
color="#204a87">(04:55:01 PM) </font></font></span><span
style="font-weight:bold;color:#204a87">dachary:
</span>dhellmann: what about this :the id of
the ressource controls the agregation of all
counters : if it is missing, all resources
of the same kind and their measures are
aggregated. Otherwise only the measures are
agreggated. <a moz-do-not-send="true"
href="http://wiki.openstack.org/EfficientMetering?action=diff&rev2=40&rev1=39"
target="_blank">http://wiki.openstack.org/EfficientMetering?action=diff&rev2=40&rev1=39</a><br>
<span style="font-weight:normal"><font><font
color="#204a87">(04:55:58 PM) </font></font></span><span
style="font-weight:bold;color:#204a87">dachary:
</span>it makes me a little unconfortable to
define such an "ad-hoc" grouping <br>
<span style="font-weight:normal"><font><font
color="#204a87">(04:56:53 PM) </font></font></span><span
style="font-weight:bold;color:#204a87">dachary:
</span>i.e. you actuall control the
aggregation by chosing which value to put in
the id column<br>
<span style="font-weight:normal"><font><font
color="#204a87">(04:58:43 PM) </font></font></span><span
style="font-weight:bold;color:#204a87">dachary:
</span>s/actuall/actually/<br>
<span style="font-weight:normal"><font><font
color="#062585">(05:05:38 PM) </font></font></span><span
style="font-weight:bold;color:#062585">***dachary
</span>reading <a moz-do-not-send="true"
href="http://www.ogf.org/documents/GFD.98.pdf"
target="_blank">http://www.ogf.org/documents/GFD.98.pdf</a><br>
<span style="font-weight:normal"><font><font
color="#204a87">(05:05:54 PM) </font></font></span><span
style="font-weight:bold;color:#204a87">dachary:
</span>I feel like we're trying to resolve a
non problem here<br>
<span style="font-weight:normal"><font><font
color="#204a87">(05:08:42 PM) </font></font></span><span
style="font-weight:bold;color:#204a87">dachary:
</span>values need to be aggregated. The raw
input is a full description of the resource
and a value ( gauge ). The question is how
to control the aggregation in a reasonably
flexible way. <br>
<span style="font-weight:normal"><font><font
color="#204a87">(05:11:34 PM) </font></font></span><span
style="font-weight:bold;color:#204a87">dachary:
</span>The definition of a counter could
probably be described as : the id of a
resource and code to fill each column
associated with it.<br>
<br>
I tried to append the following, but the
wiki kept failing. <br>
<br>
Propose that the counters are defined by a
function instead of being fixed. That helps
addressing the issue of aggregating the
bandwidth associated to a given IP into a
single counter.<br>
<br>
Alternate idea : <br>
* a counter is defined by<br>
* a name ( o1, n2, etc. ) that uniquely
identifies the nature of the measure (
outbound internet transit, amount of RAM,
etc. )<br>
* the component in which it can be found (
nova, swift etc.)<br>
* and by columns, each one is set with the
result of aggregate(find(record),record)
where<br>
* find() looks for the existing column as
found by selecting with the unique key (
maybe the name and the resource id )<br>
* record is a detailed description of the
metering event to be aggregated ( <a
moz-do-not-send="true"
href="http://wiki.openstack.org/SystemUsageData#compute.instance.exists:"
target="_blank">http://wiki.openstack.org/SystemUsageData#compute.instance.exists:</a>
)<br>
* the aggregate() function returns the
updated row. By default it just += the
counter value with the old row returned by
find()</div>
</blockquote>
<div><br>
</div>
<div>Would we want aggregation to occur within
the database where we are collecting events,
or should that move somewhere else?</div>
</div>
</div>
</blockquote>
</div>
</div>
I assume the events collected by the metering agents will
all be archived for auditing (or re-building the database)
<a moz-do-not-send="true"
href="http://wiki.openstack.org/EfficientMetering?action=diff&rev2=45&rev1=44"
target="_blank">http://wiki.openstack.org/EfficientMetering?action=diff&rev2=45&rev1=44</a><br>
<br>
Therefore the aggregation should occur when the database
is updated to account for a new event.<br>
<br>
Does this make sense ? I may have misunderstood part of
your question.<br>
</div>
</blockquote>
<div><br>
</div>
<div>I guess what I don't understand is why the aggregated
data is written back to the metering database at all. If
it's in the same database, it seems like it should be in a
different "table" (or equivalent) so the original data is
left alone.</div>
</div>
</div>
</blockquote>
In my view the events are not stored in a database, they are merely
appended to a log file. The database is built from the events with
aggregated data. I now understand that you (and Joshua Harlow) think
it's better to not aggregate the data and let the billing system do
this job. <br>
<blockquote
cite="mid:CADb+p3ScVxoddUnv4qwkc1ePkjC-o7xRzqZ_RzQO=XAcJDi+MQ@mail.gmail.com"
type="cite">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div>Maybe it's time to start focusing these discussions on
user stories?</div>
<div><br>
</div>
</div>
</div>
</blockquote>
I agree. Would you like to go first ?<br>
<br>
Cheers<br>
<br>
<pre class="moz-signature" cols="3000">--
Loïc Dachary Chief Research Officer
// eNovance labs <a class="moz-txt-link-freetext" href="http://labs.enovance.com">http://labs.enovance.com</a>
// ✉ <a class="moz-txt-link-abbreviated" href="mailto:loic@enovance.com">loic@enovance.com</a> ☎ +33 1 49 70 99 82
</pre>
</body>
</html>