Open Stack

Hey guys,

Monsyne Dragon and myself are proposing an implementation for notifications going forward. Currently my branch exists under https://code.launchpad.net/~cerberus/nova/nova_notifications. you'll see that's it been proposed for merge, but we're currently refactoring it around changes proposed at the summit during the notifications discussion, which you can see at http://etherpad.openstack.org/notifications

At the heart of the above branch is the idea that, because nova is about compute, we get notifications away from Nova as quickly as possible. As such, we've implemented a simple modular driver system which merely pushes messages out. The two sample "drivers" are for pushing messages into Rabbit, or doing nothing at all. There's been talk about adding Burrow as a third possible driver, which I don't think would be an issue.

One of the proposals is to have priority levels for each notification. What we're proposing is emulating the standard Python logging module and providing levels like "WARN' and "CRITICAL" in the notification. Additionally, the message format we're proposing will be a JSON dictionary containing the following attributes:

publisher_id - the source worker_type.host of the message.
timestamp - the GMT timestamp the notification was sent at
event_type - the literal type of event (ex. Instance Creation)
priority - patterned after the enumeration of Python logging levels in
               the set (DEBUG, WARN, INFO, ERROR, CRITICAL)
payload - A python dictionary of attributes

Message example:

    { 'publisher_id': 'compute.host1',
      'timestamp': '2011-05-09 22:00:14.621831',
      'priority': 'WARN',
      'event_type': 'compute.create_instance',
      'payload': {'instance_id': 12, ... }}

There was a lot of concern voiced over messages backing up in any of the queueing implementations, as well as the intended priority of one message over another. There are couple of immediately obvious solutions to this. We think the simplest solution is to implement N queues, where N is equal the number of priorities. Afterwards, consuming those queues is implementation specific and dependent on the solution that works best for the user.

The current plan for the Rackspace specific implementation is to use PubSubHubBub, with a dedicated worker consuming the notification queues and providing the glue necessary to work with a standard Hub implementation. I have a very immature worker implementation at https://github.com/Cerberus98/yagi if you're interested in checking that out.

We'll be going forward with this plan immediately, but we'd love feedback if you have it. Questions, comments, concerns are very much welcomed!

Matt Dietz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20110509/c85f4e9c/attachment.html>