Open Stack

During yesterday's meeting we discussed the API proposal at
http://wiki.openstack.org/EfficientMetering#API and came up with a few
missing items and other points for discussion. We should try to work out
those details on the list before the meeting next week.

The original proposal included these API calls:

GET list components
GET list components meters (argument : name of the component)
GET list [user_id|project_id|source]
GET list of meter_type
GET list of events per [user_id|project_id|source] ( allow to specify
user_id or project_id or both )
GET sum of (meter_volume, meter_duration) for meter_type and
[user_id|project_id|source]

They can be broken down into three groups:

- Discover the sorts of things the server can provide:

  - list the components providing metering data
  - list the meters for a given component
  - list the known users
  - list the known projects
  - list the known sources
  - list the types of meters known

- Fetch raw event data, without aggregation:

  - per user
  - per project
  - per source
  - per user and project

- Produce aggregate views of the data:

  - sum "volume" field for meter type over a time period
    - per user
    - per project
    - per source
  - sum "duration" field for meter type over a period of time
    - per user
    - per project
    - per source

We agreed that all of the queries related to metering data would need
to have parameters to set the start and end times with the start time
inclusive and the end time exclusive (i.e., start <= timestamp < end).

Callers also are likely to want to filter the raw data queries based
on the meter type, so we need to add that as an optional argument
there. The aggregate values have the meter type as a required argument
(because it does not make sense to aggregate data from separate meters
together).

There are a few other queries missing from that list. The items below
are based on the meeting notes and my own thoughts from yesterday,
please add to the list if I have left anything out (or suggest why we
should not implement something suggested here, of course):

- list discrete events that may not have a duration (instance
  creation, IP allocation, etc.)
- list raw event data for a resource ("what happened with a specific
  instance?")
- aggregate event data per meter type for a resource over a period of
  time ("what costs are related to this instance?")
- sum volume for meter type over a time period for a specific resource
  ("how much total bandwidth was used by a VIF?")
- sum duration for meter type over a time period for a specific
  resource ("how long did an instance run?")
- metadata for resources (such as location of instances)
- aggregating averages in addition to sums

Some of these items may be better handled in the consumer of this API
(the system that actually bills the user). I am curious to know how
others are handling auditing or other billing detail reporting for
users.

In order to support the discrete events, we need to capture
them. Perhaps those could be implemented as additional counter
types. Thoughts?

In order to provide the metadata we need to capture it. Some metadata
may change (location), so we need to support updates.

- The interesting metadata for a resource may depend on the type of
  resource. Do we need separate "tables" for that or can we normalize
  somehow?
- How do we map a resource to the correct version of its metadata at
  any given time? Timestamps seem brittle.
- Do we need to reflect the metadata in the aggregation API?

We also discussed extensions, but we should start another thread for
that topic.

Doug
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20120511/8b4f1e4c/attachment.html>