Open Stack

The email thread with the subject "[RFC] OpenStack Programming Model
Framework" from Jan 3rd covered a few ground proposals for OpenStack
projects, mainly with a focus on API. I'd like to expand on this a
bit more.

I think everyone in is in agreement that each service should default
to a REST API except for some end-user communication depending
on the service (for example, MySQL or PG protocol for database
services). Admin, provisioning, and any other gaps in functionality
not encapsulated by these application specific protocols should be
filled in via REST calls.

There is other common functionality we should consider for OpenStack
services. We don't need anything too formal, just a "best practices"
document that can change with time. This will hopefully also drive
openstack-common projects for chosen languages so we can encourage
code sharing between projects (although not required).

The main candidates for discussion are:

* Authc/authz/access - We had a marathon thread about this the other
  day, not much more to say. I think there is consensus that auth
  needs to be pluggable for all services and be flexible enough to
  accommodate various organizational structures.


* Zones and Location URIs for all objects - I think we'll all agree
  that when working with distributed services, location becomes very
  import. Issues like bandwidth limitations, latency, and location
  redundancy are top priority. Swift is already very aware, and work
  is underway in Nova, but I believe there will be a huge value in
  making a common location format across services. For example,
  being able to make requests like "launch a nova instance near
  this swift object", or if we have a queue and database services,
  "run these queue workers near a copy of this database".

  In an effort to keep things simple, I propose we simply assign URIs
  to all zones and objects across services. These would be dynamic,
  as objects can move around due to fail-over, rebalancing, and so
  forth. It would be up to the API consumer to request locations
  (with some reasonable TTL) before using them. What does this mean
  for services? Just have a 'location' attribute for every zone and
  object (Swift object, nova instances, nova volume, ...).

  The URI does imply a dotted hierarchy in order to perform suffix
  matching (for example, zone cage1.dc1.north.service.com would
  match for *.north.service.com requests). We could keep this field
  completely free-form and make the location generation/matching
  pluggable, but URIs seem to be pretty ubiquitous. DNS entries for
  these URIs are possible but not required.


* Firehouse - So far we've not discussed this too much, but I
  think when we did there was agreement that we need it. As more
  service come into the picture, we want the ability to combine and
  multiplex our logs, events, billing information, etc. so we can
  report per account, per service, and so forth. For example, as
  a user, I want to be able to see the logs or billing events with
  all entries from all my services (or filter by service), but as a
  sysadmin I may want to view per service, or per zone. We may have
  registered handlers to grab certain events for PuSH notifications
  too. To maintain maximum flexibility across deployments we need
  keep the interface generic, the payload can be a JSON object or some
  more efficient serialized message (this can be pluggable). The only
  required fields are probably:

  <timestamp> <service> <account_id> <blob>

  Where <blob> is a list of key/value pairs that handlers can
  perform routing and processing on. For a logging event, blob
  may be "priority=ERROR, message=oops!" or "priority=information,
  message=instance X launched". We can keep things really simple and
  flexible, relying on a set of documented common attributes that
  common event producers, routers, and handlers can key in on.


Thoughts?

-Eric