Open Stack

On 09/18/2014 12:31 PM, Flavio Percoco wrote:
> On 09/17/2014 10:36 PM, Joe Gordon wrote:
>> My understanding of Zaqar is that it's like SQS. SQS uses distributed
>> queues, which have a few unusual properties [0]:
>>
>>
>>      Message Order
>>
>> Amazon SQS makes a best effort to preserve order in messages, but due to
>> the distributed nature of the queue, we cannot guarantee you will
>> receive messages in the exact order you sent them. If your system
>> requires that order be preserved, we recommend you place sequencing
>> information in each message so you can reorder the messages upon receipt.
>>
>
> Zaqar guarantees FIFO. To be more precise, it does that relying on the
> storage backend ability to do so as well. Depending on the storage used,
> guaranteeing FIFO may have some performance penalties.

Would it be accurate to say that at present Zaqar does not use 
distributed queues, but holds all queue data in a storage mechanism of 
some form which may internally distribute that data among servers but 
provides Zaqar with a consistent data model of some form?

[...]
> As of now, Zaqar fully relies on the storage replication/clustering
> capabilities to provide data consistency, availability and fault
> tolerance.

Is the replication synchronous or asynchronous with respect to client 
calls? E.g. will the response to a post of messages be returned only 
once the replication of those messages is confirmed? Likewise when 
deleting a message, is the response only returned when replicas of the 
message are deleted?

> However, as far as consuming messages is concerned, it can
> guarantee once-and-only-once and/or at-least-once delivery depending on
> the message pattern used to consume messages. Using pop or claims
> guarantees the former whereas streaming messages out of Zaqar guarantees
> the later.

 From what I can see, pop provides unreliable delivery (i.e. its similar 
to no-ack). If the delete call using pop fails while sending back the 
response, the messages are removed but didn't get to the client.

What do you mean by 'streaming messages'?

[...]
> Based on our short conversation on IRC last night, I understand you're
> concerned that FIFO may result in performance issues. That's a valid
> concern and I think the right answer is that it depends on the storage.
> If the storage has a built-in FIFO guarantee then there's nothing Zaqar
> needs to do there. In the other hand, if the storage does not have a
> built-in support for FIFO, Zaqar will cover it in the driver
> implementation. In the mongodb driver, each message has a marker that is
> used to guarantee FIFO.

That marker is a sequence number of some kind that is used to provide 
ordering to queries? Is it generated by the database itself?