<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Yes, I see a general need to be able to
represent meta data the identifies associations between monitored
systems and between usage stored in the datastore. There could be
a variety of ways that we need to relate event data, so the
mechanism should be relatively generic. In addition to your use
case, we would want to be able to map instances to other tenants,
group VM's together to represent some kind of shared identity or
behavior, map instances to some kind of special service type. I
think there are big advantages to keeping the collection and
storage of this data separate:<br>
1. It does not require Nova to be aware of the details of the VM
internals. <br>
2.It allows for deferral of processing until later in the rating
process, which helps on scalability<br>
3. makes it easier to extract and report on this data for other
use cases besides the base billing<br>
4. it more extensible in the sense that you can add arbitrary
metadata without affecting the core usage data generation.<br>
<br>
We use this information as part of our rating process to determine
the correct charges to apply, so we will need to be able to query
for it.<br>
<br>
Dan<br>
<br>
On 11/5/2012 5:04 AM, Doug Hellmann wrote:<br>
</div>
<blockquote
cite="mid:CADb+p3RHzXzLpnisSE=5kCq+sBYhCv8-nuFM5GvrG_DUwFb0+Q@mail.gmail.com"
type="cite"><br>
<br>
<div class="gmail_quote">On Fri, Nov 2, 2012 at 4:42 PM, Dan Dyer
<span dir="ltr"><<a moz-do-not-send="true"
href="mailto:dan.dyer00@gmail.com" target="_blank">dan.dyer00@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>Yes, I am assuming the service controller provides a
different stream of data from the lower level VM events.
So the question is how to represent and store this
additional meta data in ceilometer. Note that there
doesn't necessarily need to be a linkage/grouping between
the resources since the association is what is actually
contained in the metadata that is provided by the service
controller. <br>
<br>
As a summary <br>
Nova provides its normal events for usage<br>
Service controller provides a mapping of nova instances to
service type and actual end user</div>
</div>
</blockquote>
<div><br>
</div>
<div>So the problem isn't necessarily that you want to measure
something different, but that the "ownership" in the existing
data is not "correct" from the perspective of the billing
system.</div>
<div><br>
</div>
<div>We have a similar issue at DreamHost. Our existing user
database has account ids that need to be mapped to tenant ids
from keystone. Rather than putting that information in
keystone, or ceilometer, we decided to store it in our system
and have the DreamHost billing system drive the ceilometer
API. Does it make sense to do something similar here?</div>
<div><br>
</div>
<div>If we definitely want ceilometer to hold the metadata, then
I could also see adding an API to let an outside system add
metadata to a resource. That would let the PaaS code, which
knows about each VM, store extra data that would be returned
with the VM metadata when a caller visits
/resources/<resourceid>.</div>
<div><br>
</div>
<div>Would you expect to be able to query using the metadata?
For example, "provide the total instance hours for all
instances with paas_tag=foo"?</div>
<div><br>
</div>
<div>Doug</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div><span class="HOEnZb"><font color="#888888"><br>
<br>
Dan</font></span>
<div>
<div class="h5"><br>
<br>
On 11/1/2012 11:25 AM, Doug Hellmann wrote:<br>
</div>
</div>
</div>
<div>
<div class="h5">
<blockquote type="cite"><br>
<br>
<div class="gmail_quote">On Thu, Nov 1, 2012 at 10:21
AM, Dan Dyer <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:dan.dyer00@gmail.com"
target="_blank">dan.dyer00@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>In some cases, the service controller is
actually running inside a VM. It would not
have access to the internals of the VM's. It
maintains its metadata separately from the
Nova infrastructure.</div>
</div>
</blockquote>
<div><br>
</div>
<div>It doesn't need internal access to the VM, but
something has to share the metadata with
ceilometer (or "join" it to the data ceilometer
has) at some point. If it would be too difficult
to get the data into the events, then it could be
done by the app that uses the ceilometer API to
query for usage. For example, the app that loads
data from ceilometer to your real billing system
could be driven by data saved by the service
controller in whatever database it uses.</div>
<div><br>
</div>
<div>Doug</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div><span><font color="#888888"><br>
<br>
DD</font></span>
<div>
<div><br>
<br>
On 10/25/2012 2:25 AM, Nick Barcet wrote:<br>
</div>
</div>
</div>
<div>
<div>
<blockquote type="cite">
<pre>Let's imagine that the service that launch instances can tag the
instance with:
a) a common service identifier (constant)
b) a uuid unique for each "Unit" of the service
such as <constant>:<uuid>
If that tag is passed onto the events which ceilometer stores in its
entirety as meta, I do not see what the difficulty would be for the
rating engine to be able to reconcile the information to handle your 2
use cases. Am I missing something?
Nick
On 10/25/2012 12:03 AM, Dan Dyer wrote:
</pre>
<blockquote type="cite">
<pre>I don't think its just a matter of adding more meters or events for a
couple of reasons:
1. In many cases the metadata I am referring to comes from a different
source than the base usage data. Nova is still emitting its normal
events, but we get the service/user mapping from a different source. I
would not characterize this data as usage metrics but more data about
the system relationships.
2. in the multiple VM case, we need to have the relationships specified
so that we can ignore the proper VM's. There has also been talk of
hybrid billing models that charge for some part of the VM usage as well
as other metrics. Once again we need a way to characterize the
relationships so that processing can associate and filter correctly.
Dan
On 10/24/2012 3:35 PM, Julien Danjou wrote:
</pre>
<blockquote type="cite">
<pre>On Wed, Oct 24 2012, Dan Dyer wrote:
</pre>
<blockquote type="cite">
<pre>Use Case 1
Service Owned Instances
There are a set of use cases where a service is acting on behalf of a
user,
the service is the owner of the VM but billing needs to be attributed
to the
end user of the system.This scenario drives two requirements:
1. Pricing is similar to base VM's but with a premium. So the type of
service for a VM needs to be identifiable so that the appropriate
pricing
can be applied.
2. The actual end user of the VM needs to be identified so usage can be
properly attributed
</pre>
</blockquote>
<pre>I think that for this, you just need to add more meters on top of the
existing one with your own user and project id information.
</pre>
<blockquote type="cite">
<pre>As an example, in some of our PAAS use cases, there is a service
controller
running on top of the base VM that maintains the control and and
manages the
customer experience. The idea is to expose the service and not have the
customer have to (or even be able to) manipulate the virtual machine
directly. So in this case, from a Nova perspective, the PAAS service
owns
the VM and it's tenantID is what is reported back in events. The way we
resolve this is to query the service controller for meta data about that
instances they own. This is stored off in a separate "table" and used to
determine the real user at aggregation time.
</pre>
</blockquote>
<pre>This is probably where you should emit the meters you need.
</pre>
<blockquote type="cite">
<pre>Use Case 2
Multple Instances combine to make a billable "product/service"
In this use case, a service might consist of several VM's, but the
actual
number does not directly drive the billing. An example of this might
be a
redundant service that has a primary and two backup VM's that make up a
deployment. The customer is charged for the service, not the fact
that there
are 3 VM's running. Once again, we need meta data that is able to
describe
this relationship so that when the billing records are processed, this
relationship can be identified and billed properly.
</pre>
</blockquote>
<pre>Kind of the same here, if you don't want to really bill the vm, just
don't meter them (or ignore the meters) and emit your own meter via your
PaaS platform to bill your customer.
Or is there a limitation I miss?
</pre>
</blockquote>
<pre>_______________________________________________
Mailing list: <a moz-do-not-send="true" href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a>
Post to : <a moz-do-not-send="true" href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a>
Unsubscribe : <a moz-do-not-send="true" href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a>
More help : <a moz-do-not-send="true" href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a>
</pre>
</blockquote>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Mailing list: <a moz-do-not-send="true" href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a>
Post to : <a moz-do-not-send="true" href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a>
Unsubscribe : <a moz-do-not-send="true" href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a>
More help : <a moz-do-not-send="true" href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a>
</pre>
</blockquote>
<br>
</div>
</div>
</div>
<br>
_______________________________________________<br>
Mailing list: <a moz-do-not-send="true"
href="https://launchpad.net/%7Eopenstack"
target="_blank">https://launchpad.net/~openstack</a><br>
Post to : <a moz-do-not-send="true"
href="mailto:openstack@lists.launchpad.net"
target="_blank">openstack@lists.launchpad.net</a><br>
Unsubscribe : <a moz-do-not-send="true"
href="https://launchpad.net/%7Eopenstack"
target="_blank">https://launchpad.net/~openstack</a><br>
More help : <a moz-do-not-send="true"
href="https://help.launchpad.net/ListHelp"
target="_blank">https://help.launchpad.net/ListHelp</a><br>
<br>
</blockquote>
</div>
<br>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</blockquote>
<br>
</body>
</html>