Open Stack

On 22/09/14 22:04, Joe Gordon wrote:
> To me this is less about valid or invalid choices. The Zaqar team is
> comparing Zaqar to SQS, but after digging into the two of them, zaqar
> barely looks like SQS. Zaqar doesn't guarantee what IMHO is the most
> important parts of SQS: the message will be delivered and will never be
> lost by SQS.

I agree that this is the most important feature. Happily, Flavio has 
clarified this in his other thread[1]:

  *Zaqar's vision is to provide a cross-cloud interoperable,
   fully-reliable messaging service at scale that is both, easy and not
   invasive, for deployers and users.*

   ...

   Zaqar aims to be a fully-reliable service, therefore messages should
   never be lost under any circumstances except for when the message's
   expiration time (ttl) is reached

So Zaqar _will_ guarantee reliable delivery.

> Zaqar doesn't have the same scaling properties as SQS.

This is true. (That's not to say it won't scale, but it doesn't scale in 
exactly the same way that SQS does because it has a different architecture.)

It appears that the main reason for this is the ordering guarantee, 
which was introduced in response to feedback from users. So this is 
clearly a different design choice: SQS chose reliability plus 
effectively infinite scalability, while Zaqar chose reliability plus 
FIFO. It's not feasible to satisfy all three simultaneously, so the 
options are:

1) Implement two separate modes and allow the user to decide
2) Continue to choose FIFO over infinite scalability
3) Drop FIFO and choose infinite scalability instead

This is one of the key points on which we need to get buy-in from the 
community on selecting one of these as the long-term strategy.

> Zaqar is aiming for low latency per message, SQS doesn't appear to be.

I've seen no evidence that Zaqar is actually aiming for that. There are 
waaay lower-latency ways to implement messaging if you don't care about 
durability (you wouldn't do store-and-forward, for a start). If you see 
a lot of talk about low latency, it's probably because for a long time 
people insisted on comparing Zaqar to RabbitMQ instead of SQS.

(Let's also be careful not to talk about high latency as if it were a 
virtue in itself; it's simply something we would happily trade off for 
other properties. Zaqar _is_ making that trade-off.)

> So if Zaqar isn't SQS what is Zaqar and why should I use it?

If you are a small-to-medium user of an SQS-like service, Zaqar is like 
SQS but better because not only does it never lose your messages but 
they always arrive in order, and you have the option to fan them out to 
multiple subscribers. If you are a very large user along one particular 
dimension (I believe it's number of messages delivered from a single 
queue, but probably Gordon will correct me :D) then Zaqar may not _yet_ 
have a good story for you.

cheers,
Zane.

[1] 
http://lists.openstack.org/pipermail/openstack-dev/2014-September/046809.html