Open Stack

> A fully connected graph is needed, which means a single publisher
> won't suffiice.  This means that each node who wishes to receive
> broadcasts must connect to all publishers.  In most fully
> decentrialized systems I've worked in, there is a notion of a quorum
> of network membership, and this is generally a queryable or locally
> maintained (and fully connected) service.


I'm the primary author of the ZeroMQ driver.

ZeroMQ doesn't provide a quorum or any such services by itself. Thus, the
lack of a name service / membership solution was greatly a matter of scope.
I feel that having a fully connected, synchronous solution is really not
necessary. If you look at the CAP theorem and applied it to messaging,
Zookeeper would be on the side of consistency, and I don't think we really
need that... We do need availability and partition tolerance. However, with
the matchmaker design, we could bolt onto a service that provided
consistency, if desired.

The concept of the matchmaker is such that the queryable service is
abstracted away. In fact, with the ServiceGroup API feature which has
recently landed in Grizzly, we now have MySQL and Zookeeper backends that
can power (or replace) the MatchMaker. One of my goals for Grizzly is to
bridge these features, either by replacing the matchmaker outright, or
creating a backend dependent on the ServiceGroup API.




Why the distinction?  I guess it only matters if you're interested in
> adding and removing participating nodes at run time, especially nodes
> which might not be performing all openstack tasks, but only a subset.
> Then it is up to only that node to subscribe to the topics that he is
> after.  In the Matchmaker approach, a node that is joining the network
> would be responsible for ensuring that all local matchmakers are
> updated with the topics in which it is interested.
>

Keep in mind we do need to maintain compatibility with AMQP, especially
since just about everyone is actually using RabbitMQ (but there ARE people
using the ZeroMQ driver).  With Rabbit, the node that joins creates its own
topic and in subscribing to any shared topics for which it receives
messages. I'm actually okay with this pattern, although it does mean we
need a dynamic membership solution as soon as we can get it. What I
implemented for Folsom was entirely static, but it was suitable for Nova's
broadcast pattern where there were very small and fairly static sets of
consumers of those broadcast messages (namely: schedulers).

-- 
Regards,
Eric Windisch
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20121222/f948fd54/attachment.html>