Open Stack

Russell Bryant  wrote:
> So, it seems that at the root of this, you're looking for a
> cloud-compatible way for instances to message each other.

No: discovery of peers, not messaging.  After discovery, communication
between nodes will then be done directly e.g. over TCP.  Examples of
services that work using this model:  Elasticsearch, JBoss Data Grid,
anything using JGroups, the next version of Zookeeper, etc.  The
instances just need some way to find each other; a nice way to think
of this is as a replacement for multicast-discovery on the cloud.

All these services then switch to direct messaging, because using an
intermediate service introduces too much latency.

With this blueprint though, we could build and run a great backend for
Marconi, using OOO.

> I really don't see the metadata API as the appropriate place for that.

But I presume you're OK with it for discovery? :-)

> How about using Marconi here?  If not, what's missing from Marconi's API
> to solve your messaging use case to allow instances to discover each other?

Well, again: discovery, so Marconi isn't the natural fit it may at
first appear.  Not sure if Marconi supports 'broadcast' queues (that
would be the missing piece if it doesn't).  But, even if we could
abuse a Marconi queue for this:

1) Marconi isn't widely deployed
2) There is no easy way for a node to discover Marconi, even if it was deployed.
3) There is no easy way for a node to authenticate to Marconi, even if
we could discover it

I absolutely think we should fix each of those obstacles, and I'm sure
we will eventually.  But in the meantime, let's get this into
Icehouse!

Justin