<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#ffffff">
On 04/24/2012 03:06 PM, Monsyne Dragon wrote:
<blockquote
cite="mid:70A19286-1CB4-4EF0-921F-9B7C197F90F6@rackspace.com"
type="cite">
<pre wrap="">Yes, we emit bandwidth (bytes in/out) on a per VIF basis from each instance The event has the somewhat generic name of 'compute.instance.exists' and is emitted on an periodic basis, currently by a cronjob.
Currently, we only populate bandwidth data from XenServer, but if the hook is implemented for the kvm, etc drivers, it will be picked up automatically for them as well.
Note that we could report other metrics similarly.
</pre>
</blockquote>
Hi,<br>
<br>
Thanks for clarifying this. So you're suggesting that the metering
agent should collect this data from the nova queue instead of
extracting it from the system (interface, disk stats etc.) ? And for
other openstack components ( as Nick Barcet suggests below ) the
metering agent will have to find another way. Or do you have
something else in mind ?<br>
<br>
Cheers<br>
<br>
On 04/24/2012 12:17 PM, Nick Barcet wrote:
<blockquote cite="mid:4F967D9E.10501@canonical.com" type="cite">
<div class="moz-text-plain" wrap="true" style="font-family:
-moz-fixed; font-size: 12px;" lang="x-western">
<pre wrap="">On 04/23/2012 10:45 PM, Doug Hellmann wrote:
</pre>
<blockquote type="cite" style="color: rgb(0, 0, 0);">
<pre wrap=""><span class="moz-txt-citetags">> </span>
<span class="moz-txt-citetags">> </span>
<span class="moz-txt-citetags">> </span>On Mon, Apr 23, 2012 at 4:14 PM, Brian Schott
<span class="moz-txt-citetags">> </span><<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:brian.schott@nimbisservices.com">brian.schott@nimbisservices.com</a>
<span class="moz-txt-citetags">> </span><a moz-do-not-send="true" class="moz-txt-link-rfc2396E" href="mailto:brian.schott@nimbisservices.com"><mailto:brian.schott@nimbisservices.com></a>> wrote:
<span class="moz-txt-citetags">> </span>
<span class="moz-txt-citetags">> </span> Doug,
<span class="moz-txt-citetags">> </span>
<span class="moz-txt-citetags">> </span> Do we mirror the table structure of nova, etc. and add
<span class="moz-txt-citetags">> </span> created/modified columns?
<span class="moz-txt-citetags">> </span>
<span class="moz-txt-citetags">> </span>
<span class="moz-txt-citetags">> </span> Or do we flatten into an instance event record with everything?
<span class="moz-txt-citetags">> </span>
<span class="moz-txt-citetags">> </span>
<span class="moz-txt-citetags">> </span>I lean towards flattening the data as it is recorded and making a second
<span class="moz-txt-citetags">> </span>pass during the bill calculation. You need to record instance
<span class="moz-txt-citetags">> </span>modifications separately from the creation, especially if the
<span class="moz-txt-citetags">> </span>modification changes the billing rate. So you might have records for:
<span class="moz-txt-citetags">> </span>
<span class="moz-txt-citetags">> </span>created instance, with UUID, name, size, timestamp, ownership
<span class="moz-txt-citetags">> </span>information, etc.
<span class="moz-txt-citetags">> </span>resized instance, with UUID, name, new size, timestamp, ownership
<span class="moz-txt-citetags">> </span>information, etc.
<span class="moz-txt-citetags">> </span>deleted instance, with UUID, name, size, timestamp, ownership
<span class="moz-txt-citetags">> </span>information, etc.
<span class="moz-txt-citetags">> </span>
<span class="moz-txt-citetags">> </span>Maybe some of those values don't need to be reported in some cases, but
<span class="moz-txt-citetags">> </span>if you record a complete picture of the state of the instance then the
<span class="moz-txt-citetags">> </span>code that aggregates the event records to produce billing information
<span class="moz-txt-citetags">> </span>can use it to make decisions about how to record the charges.
<span class="moz-txt-citetags">> </span>
<span class="moz-txt-citetags">> </span>There is also the case where an instance is still no longer running but
<span class="moz-txt-citetags">> </span>nova thinks it is (or the reverse), so some sort of auditing sweep needs
<span class="moz-txt-citetags">> </span>to be included (I think that's what Dough called the "farmer" but I
<span class="moz-txt-citetags">> </span>don't have my notes in front of me).
</pre>
</blockquote>
<pre wrap="">When I wrote [1], one of the things that I never assumed was how agents
would collect their information. I imagined that the system should allow
for multiple implementation of agents that would collect the same
counters, assuming that 2 implementations for the same counter should
never be running at once.
That said, I am not sure an event based collection of what nova is
notifying would satisfy the requirements I have heard from many cloud
providers:
- how do we ensure that event are not forged or lost in the current nova
system?
- how can I be sure that an instance has not simply crashed and never
started?
- how can I collect information which is not captured by nova events?
Hence the proposal to use a dedicated event queue for billing, allowing
for agents to collect and eventually validate data from different
sources, including, but not necessarily limiting, collection from the
nova events.
Moreover, as soon as you generalize the problem to other components than
just Nova (swift, glance, quantum, daas, ...) just using the nova event
queue is not an option anymore.
[1] <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://wiki.openstack.org/EfficientMetering">http://wiki.openstack.org/EfficientMetering</a>
Nick
</pre>
</div>
<br>
</blockquote>
<br>
<blockquote
cite="mid:70A19286-1CB4-4EF0-921F-9B7C197F90F6@rackspace.com"
type="cite">
<pre wrap="">
On Apr 24, 2012, at 6:20 AM, Sandy Walsh wrote:
</pre>
<blockquote type="cite">
<pre wrap="">I think we have support for this currently in some fashion, Dragon?
-S
On 04/24/2012 12:55 AM, Loic Dachary wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Metering needs to account for the "volume of data sent to external network destinations " ( i.e. n4 in <a class="moz-txt-link-freetext" href="http://wiki.openstack.org/EfficientMetering">http://wiki.openstack.org/EfficientMetering</a> ) or the disk I/O etc. This kind of resource is billable.
The information described at <a class="moz-txt-link-freetext" href="http://wiki.openstack.org/SystemUsageData">http://wiki.openstack.org/SystemUsageData</a> will be used by metering but other data sources need to be harvested as well.
</pre>
</blockquote>
</blockquote>
<pre wrap="">
--
Monsyne M. Dragon
OpenStack/Nova
cell 210-441-0965
work x 5014190
_______________________________________________
Mailing list: <a class="moz-txt-link-freetext" href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a>
Post to : <a class="moz-txt-link-abbreviated" href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a>
Unsubscribe : <a class="moz-txt-link-freetext" href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a>
More help : <a class="moz-txt-link-freetext" href="https://help.launchpad.net/ListHelp">https://help.launchpad.net/ListHelp</a>
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="3000">--
Loïc Dachary Chief Research Officer
// eNovance labs <a class="moz-txt-link-freetext" href="http://labs.enovance.com">http://labs.enovance.com</a>
// ✉ <a class="moz-txt-link-abbreviated" href="mailto:loic@enovance.com">loic@enovance.com</a> ☎ +33 1 49 70 99 82
</pre>
</body>
</html>