Open Stack

Excerpts from Thierry Carrez's message of 2014-09-12 02:16:42 -0700:
> Clint Byrum wrote:
> > Excerpts from Flavio Percoco's message of 2014-09-11 04:14:30 -0700:
> >> Is Zaqar being optimized as a *queuing* service? I'd say no. Our goal is
> >> to optimize Zaqar for delivering messages and supporting different
> >> messaging patterns.
> > 
> > Awesome! Just please don't expect people to get excited about it for
> > the lighter weight queueing workloads that you've claimed as use cases.
> > 
> > I totally see Horizon using it to keep events for users. I see Heat
> > using it for stack events as well. I would bet that Trove would benefit
> > from being able to communicate messages to users.
> > 
> > But I think in between Zaqar and the backends will likely be a lighter
> > weight queue-only service that the users can just subscribe to when they
> > don't want an inbox. And I think that lighter weight queue service is
> > far more important for OpenStack than the full blown random access
> > inbox.
> > 
> > I think the reason such a thing has not appeared is because we were all
> > sort of running into "but Zaqar is already incubated". Now that we've
> > fleshed out the difference, I think those of us that need a lightweight
> > multi-tenant queue service should add it to OpenStack.  Separately. I hope
> > that doesn't offend you and the rest of the excellent Zaqar developers. It
> > is just a different thing.
> > 
> >> Should we remove all the semantics that allow people to use Zaqar as a
> >> queue service? I don't think so either. Again, the semantics are there
> >> because Zaqar is using them to do its job. Whether other folks may/may
> >> not use Zaqar as a queue service is out of our control.
> >>
> >> This doesn't mean the project is broken.
> > 
> > No, definitely not broken. It just isn't actually necessary for many of
> > the stated use cases.
> 
> Clint,
> 
> If I read you correctly, you're basically saying the Zaqar is overkill
> for a lot of people who only want a multi-tenant queue service. It's
> doing A+B. Why does that prevent people who only need A from using it ?
> 
> Is it that it's actually not doing A well, from a user perspective ?
> Like the performance sucks, or it's missing a key primitive ?
> 
> Is it that it's unnecessarily complex to deploy, from a deployer
> perspective, and that something only doing A would be simpler, while
> covering most of the use cases?
> 
> Is it something else ?
> 
> I want to make sure I understand your objection. In the "user
> perspective" it might make sense to pursue both options as separate
> projects. In the "deployer perspective" case, having a project doing A+B
> and a project doing A doesn't solve anything. So this affects the
> decision we have to take next Tuesday...

I believe that Zaqar does two things, inbox semantics, and queue
semantics. I believe the queueing is a side-effect of needing some kind
of queue to enable users to store and subscribe to messages in the
inbox.

What I'd rather see is an API for queueing, and an API for inboxes
which integrates well with the queueing API. For instance, if a user
says "give me an inbox" I think Zaqar should return a queue handle for
sending into the inbox the same way Nova gives you a Neutron port if
you don't give it one. You might also ask for a queue to receive push
messages from the inbox. Point being, the queues are not the inbox,
and the inbox is not the queues.

However, if I just want a queue, just give me a queue. Don't store my
messages in a randomly addressable space, and don't saddle the deployer
with the burden of such storage. Put the queue API in front of a scalable
message queue and give me a nice simple HTTP API. Users would likely be
thrilled. Heat, Nova, Ceilometer, probably Trove and Sahara, could all
make use of just this. Only Horizon seems to need a place to keep the
messages around while users inspect them.

Whether that is two projects, or one, separation between the two API's,
and thus two very different types of backends, is something I think
will lead to more deployers wanting to deploy both, so that they can
bill usage appropriately and so that their users can choose wisely.