Open Stack

On Wed, Apr 24 2013, Eric Windisch wrote:

> It gets tricky because ZeroMQ doesn't have "a connection", although I
> suppose it could have different connection profiles (a different port
> number, for instance).

Well, a "connection" could be something specific to the implementation.
- RabbitMQ (anq qpid I guess) would be host/port/control-exchange
- ZMQ would be host/port

And each project would use a separate connection to communicate.

> We have been discussing specifying a message destination as part of
> call/cast instead of topic as presently sent.
>
> The new API might be something akin to:
>
> dest = rpc.make_destination('compute', host='compute1', cell=CellState)
>         rpc.cast(context, dest, msg)
>
> # This would replace rpc.cast(context, topic, msg)
>         # which would have a check to set topic = dest if not isinstance(RpcDestination, dest)

That sounds like a good move and handles transition nicely.
I'm not sure about the make_destination() prototype however -- but that
doesn't worry me for now.

> Presently, we have various ways of specifying the location of the queues:
>  * rabbit_host + rabbit_port + control_exchange (plus auth)
>  * qpid_host + qpid_port + control_exchange (plus auth)
>
>
>  * zmq_port + (matchmaker || consumer_host)
>  * cell (which maps to some subset of the above)
>
> This makes figuring out the proper abstraction for make_destination()
> a bit complex if we want to do anything besides topic and host.

I see that.

> The ZeroMQ example above is a bit strange, but I wasn't sure how else
> to express it. Basically, we need to lookup some topics in the
> matchmaker, but if we're sending messages to a host directly, we can
> skip that. The code presently knows to send messages to a host by
> delimiting on a period, but it turns out that projects are suddenly
> using periods in topics to delimitate all sorts of other things.

This is something that could probably be improved in a new version of
the API I presume.

> For ZeroMQ, a control_exchange is effectively a zmq_port *AND* its
> associated matchmaker, which is pluggable. Plugins usually will have
> their own host/port/auth. The MatchMaker basically maintains a mapping
> of topics to consuming peers. Technically, we could lookup all topics
> here, even "compute.host" topics, but we can circumvent a global
> lookup by sending messages directly to "host".

Understood (because well explained, thanks :)

> Perhaps an rpc-abstraction level concept of a "cell" or "project" is
> needed? That would be an identifier encompassing all the connection
> details (i.e. host/control_exchange, zmq_port/matchmaker), however
> such may be defined for the driver being used?

That sounds like a good idea to me. That's what I called "connection" in
my previous mail actually. This may be just an abstract class
implemented by each driver in its own way.

-- 
Julien Danjou
# Free Software hacker # freelance consultant
# http://julien.danjou.info
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130425/6e94ec58/attachment.pgp>