Open Stack

On 09/18/2014 04:09 PM, Gordon Sim wrote:
> On 09/18/2014 12:31 PM, Flavio Percoco wrote:
>> On 09/17/2014 10:36 PM, Joe Gordon wrote:
>>> My understanding of Zaqar is that it's like SQS. SQS uses distributed
>>> queues, which have a few unusual properties [0]:
>>>
>>>
>>>      Message Order
>>>
>>> Amazon SQS makes a best effort to preserve order in messages, but due to
>>> the distributed nature of the queue, we cannot guarantee you will
>>> receive messages in the exact order you sent them. If your system
>>> requires that order be preserved, we recommend you place sequencing
>>> information in each message so you can reorder the messages upon
>>> receipt.
>>>
>>
>> Zaqar guarantees FIFO. To be more precise, it does that relying on the
>> storage backend ability to do so as well. Depending on the storage used,
>> guaranteeing FIFO may have some performance penalties.
> 
> Would it be accurate to say that at present Zaqar does not use
> distributed queues, but holds all queue data in a storage mechanism of
> some form which may internally distribute that data among servers but
> provides Zaqar with a consistent data model of some form?

I think this is accurate. The queue's distribution depends on the
storage ability to do so and deployers will be able to choose what
storage works best for them based on this as well. I'm not sure how
useful this separation is from a user perspective but I do see the
relevance when it comes to implementation details and deployments.


> [...]
>> As of now, Zaqar fully relies on the storage replication/clustering
>> capabilities to provide data consistency, availability and fault
>> tolerance.
> 
> Is the replication synchronous or asynchronous with respect to client
> calls? E.g. will the response to a post of messages be returned only
> once the replication of those messages is confirmed? Likewise when
> deleting a message, is the response only returned when replicas of the
> message are deleted?

It depends on the driver implementation and/or storage configuration.
For example, in the mongodb driver, we use the default write concern
called "acknowledged". This means that as soon as the message gets to
the master node (note it's not written on disk yet nor replicated) zaqar
will receive a confirmation and then send the response back to the
client. This is also configurable by the deployer by changing the
default write concern in the mongodb uri using `?w=SOME_WRITE_CONCERN`[0].

[0] http://docs.mongodb.org/manual/reference/connection-string/#uri.w

> 
>> However, as far as consuming messages is concerned, it can
>> guarantee once-and-only-once and/or at-least-once delivery depending on
>> the message pattern used to consume messages. Using pop or claims
>> guarantees the former whereas streaming messages out of Zaqar guarantees
>> the later.
> 
> From what I can see, pop provides unreliable delivery (i.e. its similar
> to no-ack). If the delete call using pop fails while sending back the
> response, the messages are removed but didn't get to the client.

Correct, pop works like no-ack. If you want to have pop+ack, it is
possible to claim just 1 message and then delete it.

> 
> What do you mean by 'streaming messages'?

I'm sorry, that went out wrong. I had the "browsability" term in my head
and went with something even worse. By streaming messages I meant
polling messages without claiming them. In other words, at-least-once is
guaranteed by default, whereas once-and-only-once is guaranteed just if
claims are used.

> 
> [...]
>> Based on our short conversation on IRC last night, I understand you're
>> concerned that FIFO may result in performance issues. That's a valid
>> concern and I think the right answer is that it depends on the storage.
>> If the storage has a built-in FIFO guarantee then there's nothing Zaqar
>> needs to do there. In the other hand, if the storage does not have a
>> built-in support for FIFO, Zaqar will cover it in the driver
>> implementation. In the mongodb driver, each message has a marker that is
>> used to guarantee FIFO.
> 
> That marker is a sequence number of some kind that is used to provide
> ordering to queries? Is it generated by the database itself?

It's a sequence number to provide ordering to queries, correct.
Depending on the driver, it may be generated by Zaqar or the database.
In mongodb's case it's generated by Zaqar[0].

[0]
https://github.com/openstack/zaqar/blob/master/zaqar/queues/storage/mongodb/queues.py#L103-L185

-- 
@flaper87
Flavio Percoco