Open Stack

On Thu, Sep 18, 2014 at 7:45 AM, Flavio Percoco <flavio at redhat.com> wrote:
> On 09/18/2014 04:09 PM, Gordon Sim wrote:
>> On 09/18/2014 12:31 PM, Flavio Percoco wrote:
>>> Zaqar guarantees FIFO. To be more precise, it does that relying on the
>>> storage backend ability to do so as well. Depending on the storage used,
>>> guaranteeing FIFO may have some performance penalties.
>>
>> Would it be accurate to say that at present Zaqar does not use
>> distributed queues, but holds all queue data in a storage mechanism of
>> some form which may internally distribute that data among servers but
>> provides Zaqar with a consistent data model of some form?
>
> I think this is accurate. The queue's distribution depends on the
> storage ability to do so and deployers will be able to choose what
> storage works best for them based on this as well. I'm not sure how
> useful this separation is from a user perspective but I do see the
> relevance when it comes to implementation details and deployments.

Guaranteeing FIFO and not using a distributed queue architecture
*above* the storage backend are both scale-limiting design choices.
That Zaqar's scalability depends on the storage back end is not a
desirable thing in a cloud-scale messaging system in my opinion,
because this will prevent use at scales which can not be accommodated
by a single storage back end.

And based on my experience consulting for companies whose needs grew
beyond the capabilities of a single storage backend, moving to
application-aware sharding required a significant amount of
rearchitecture in most cases.

-Devananda