Open Stack

>  
> What about making all of those arguments except topic part of the constructor for a Connection, and just instantiating more than one of them to talk to different services?
If we can figure out what they are, or how to make this per-driver and store them in a reasonable way - I think we might be on the right path.  
>  
>  
>  
> It seems like the host and port options for rabbit and qpid map to the same values for the matchmaker, don't they? The fact that the actual message doesn't go to the matchmaker is an implementation detail of ZMQ that it can hide.
The matchmaker is pluggable. Some matchmakers use a flat file or local database, not a network service (although this may practically become a requirement for a network service, in the long run?).  There may not be a single IP or DNS based endpoint for this, it may be multiple hosts, or a multicast address for communications, etc. I think it is premature to suggest that all the backends will have a host/port/user/password associated, and that other configuration fields won't be necessary.

Also, I think that at least optionally, the matchmaker might become tied to the AMQP drivers. That might be a good thing, at least in leveling the field. The more we talk about message encryption, and in some models, even for signing, we might find having the matchmaker advantageous. TBD.
> Right, it seems like the host portion needs to be passed as a separate argument and the AMQP drivers can combine it with the topic, instead of going the other way around.
I'm suggesting we let "service.host" be a valid topic, but *also* pass the host variable into the call/cast so that drivers can do what they will with it. It won't be *necessary* for the topic to be called "service.host", but it won't be illegal.

> > > For ZeroMQ, a control_exchange is effectively a zmq_port *AND* its
> > > associated matchmaker, which is pluggable. Plugins usually will have
> > > their own host/port/auth. The MatchMaker basically maintains a mapping
> > > of topics to consuming peers. Technically, we could lookup all topics
> > > here, even "compute.host" topics, but we can circumvent a global
> > > lookup by sending messages directly to "host".
> >  
> >  
> > Understood (because well explained, thanks :)
>  
> How does the ZMQ driver know where the host is without asking the matchmaker? Is that "host" value assumed to be resolvable via DNS?
Either because it infers it from the call/cast command (presently by inspecting the topic - which is no longer a valid source of authority), or if there is no host to be inferred, by looking up in the matchmaker.

The host itself is whatever is passed as CONF.rpc_zmq_host, which MUST match whatever Nova's CONF.host is set to. This defaults to the FQDN, such that DNS resolution is required. In *practice*, we override both of these variables to the system's IP address (for the interface on which messaging should happen)
>  
>  
> > > Perhaps an rpc-abstraction level concept of a "cell" or "project" is
> > > needed? That would be an identifier encompassing all the connection
> > > details (i.e. host/control_exchange, zmq_port/matchmaker), however
> > > such may be defined for the driver being used?
> >  
> >  
> > That sounds like a good idea to me. That's what I called "connection" in
> > my previous mail actually. This may be just an abstract class
> > implemented by each driver in its own way.
>  
>  
> What defines a "cell" for RPC? Just the alternate host and port for rabbit, qpid, or matchmaker? If so, then the caller just needs to create a new Connection with the right settings, and we don't need to have the concept of "cell" leak into the RPC library API at all. It's there now because we're using a global connection object, but sometimes want to talk to something different. If we stop having a global, the awkwardness will go away and the API can be simpler.  
You have this basically correct. Part of the problem is that we don't have that abstraction currently, and the parameters passed from the cell code for host/etc are really ill-matching for the MatchMaker requirements… and confused with the unclear separation of zmq_port and control_exchange for ZeroMQ -- and a question if ZeroMQ messages should, or should not, be brokered through to child cells (which may be necessary for some network security purposes, but unnecessary for queue scaling)

What I'd like to have, ideally, is the cell configuration essentially reconfigure the ZeroMQ settings as necessary, per the deployer's preference. A brokered-to-cell solution wouldn't be available, at least not presently, but such a design would allow that to be a possible feature addition/request, should it be requested.  (I also don't want to bike shed or overdesign too much on what people *might* do or want down the road)

Regards,
Eric Windisch