Open Stack

On 23/09/14 19:29, Devananda van der Veen wrote:
> On Mon, Sep 22, 2014 at 5:47 PM, Zane Bitter <zbitter at redhat.com> wrote:
>> On 22/09/14 17:06, Joe Gordon wrote:
>>>
>>> If 50,000 messages per second doesn't count as small-to-moderate then
>>> Zaqar
>>> does not fulfill a major SQS use case.
>>
>>
>> It's not a drop-in replacement, but as I mentioned you can recreate the SQS
>> semantics exactly *and* get the scalability benefits of that approach by
>> sharding at the application level and then round-robin polling.
>>
>
> This response seems dismissive to application developers deciding what
> cloud-based messaging system to use for their application.
>
> If I'm evaluating two messaging services, and one of them requires my
> application to implement autoscaling and pool management, and the
> other does not, I'm clearly going to pick the one which makes my
> application development *simpler*.

This is absolutely true, but the point I was trying to make earlier in 
the thread is that for other use cases you can make exactly the same 
argument going in the other direction: if I'm evaluating two messaging 
services, and one of them requires my application to implement 
reordering of messages by sequence number, and the other does not, I'm 
clearly going to pick the one which makes my application development 
*simpler*.

So it's not a question of "do we make developers do more work?". It's a 
question of "*which* developers do we make do more work?".

> Also, choices made early in a
> product's lifecycle (like, say, a facebook game) about which
> technology they use (like, say, for messaging) are often informed by
> hopeful expectations of explosive growth and fame.
>
> So, based on what you've said, if I were a game developer comparing
> SQS and Zaqar today, it seems clear that, if I picked Zaqar, and my
> game gets really popular, it's also going to have to be engineered to
> handle autoscaling of queues in Zaqar. Based on that, I'm going to
> pick SQS. Because then I don't have to worry about what I'm going to
> do when my game has 100 million users and there's still just one
> queue.

I totally agree, and that's why I'm increasingly convinced that Zaqar 
should eventually offer the choice of either. Happily, thanks to the 
existence of Flavours, I believe this can be implemented in the future 
as an optional distribution layer *above* the storage back end without 
any major changes to the current API or architecture. (One open 
question: would this require dropping browsability from the API?)

The key question here is if we're satisfied with the possibility of 
adding this in the future, or if we want to require Zaqar to dump the 
users with the in-order use case in favour of the users with the 
massive-scale use case. If we wanted that then a major re-think would be 
in order.

cheers,
Zane.