Open Stack

On 09/24/2014 12:06 AM, Joe Gordon wrote:
> 
> 
> On Tue, Sep 23, 2014 at 2:40 AM, Flavio Percoco <flavio at redhat.com
> <mailto:flavio at redhat.com>> wrote:
> 
>     On 09/23/2014 05:13 AM, Clint Byrum wrote:
>     > Excerpts from Joe Gordon's message of 2014-09-22 19:04:03 -0700:
> 
>     [snip]
> 
>     >>
>     >> To me this is less about valid or invalid choices. The Zaqar team is
>     >> comparing Zaqar to SQS, but after digging into the two of them, zaqar
>     >> barely looks like SQS. Zaqar doesn't guarantee what IMHO is the most
>     >> important parts of SQS: the message will be delivered and will never be
>     >> lost by SQS. Zaqar doesn't have the same scaling properties as SQS. Zaqar
>     >> is aiming for low latency per message, SQS doesn't appear to be. So if
>     >> Zaqar isn't SQS what is Zaqar and why should I use it?
>     >>
>     >
>     > I have to agree. I'd like to see a simple, non-ordered, high latency,
>     > high scale messaging service that can be used cheaply by cloud operators
>     > and users. What I see instead is a very powerful, ordered, low latency,
>     > medium scale messaging service that will likely cost a lot to scale out
>     > to the thousands of users level.
> 
>     I don't fully agree :D
> 
>     Let me break the above down into several points:
> 
>     * Zaqar team is comparing Zaqar to SQS: True, we're comparing to the
>     *type* of service SQS is but not *all* the guarantees it gives. We're
>     not working on an exact copy of the service but on a service capable of
>     addressing the same use cases.
> 
>     * Zaqar is not guaranteeing reliability: This is not true. Yes, the
>     current default write concern for the mongodb driver is `acknowledge`
>     but that's a bug, not a feature [0] ;)
> 
>     * Zaqar doesn't have the same scaling properties as SQS: What are SQS
>     scaling properties? We know they have a big user base, we know they have
>     lots of connections, queues and what not but we don't have numbers to
>     compare ourselves with.
> 
>  
> Here is *a* number
> 30k messages per second on a single
> queue: http://java.dzone.com/articles/benchmarking-sqs

I know how to get those links and I had read them before. For example,
here's[0] a 2 years older one that tests a different scenario and has a
quite different result.

My point is that it's not as easy as to say "X doesn't scale as Y". We
know, based on Zaqar's architecture, that depending on the storage there
are some scaling limits the service could hit but without more (or
proper) load tests I think that's just an assumption based on what we
know about the service architecture and not the storage itself. There
are benchmarks about mongodb but it'd be unfair to use those as the
definitive reference since the schema plays a huge role there.

And with this, I'm neither saying Zaqar scales unlimitedly regardless of
the storage backend nor that there are no limits at all. I'm aware
there's lot to improve in the service.

[0]
http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/throughput.html

Thanks for sharing,
Flavio

>  
> 
> 
>     * Zaqar is aiming for low latency per message: This is not true and I'd
>     be curious to know where did this come from. A couple of things to
>     consider:
> 
>             - First and foremost, low latency is a very relative
>     measure  and it
>     depends on each use-case.
>             - The benchmarks Kurt did were purely informative. I believe
>     it's good
>     to do them every once in a while but this doesn't mean the team is
>     mainly focused on that.
>             - Not being focused on 'low-latency' does not mean the team will
>     overlook performance.
> 
>     * Zaqar has FIFO and SQS doesn't: FIFO won't hurt *your use-case* if
>     ordering is not a requirement but the lack of it does when ordering is a
>     must.
> 
>     * Scaling out Zaqar will cost a lot: In terms of what? I'm pretty sure
>     it's not for free but I'd like to understand better this point and
>     figure out a way to improve it, if possible.
> 
>     * If Zaqar isn't SQS then what is it? Why should I use it?: I don't
>     believe Zaqar is SQS as I don't believe nova is EC2. Do they share
>     similar features and provide similar services? Yes, does that mean you
>     can address similar use cases, hence a similar users? Yes.
> 
>     In addition to the above, I believe Zaqar is a simple service, easy to
>     install and to interact with. From a user perspective the semantics are
>     few and the concepts are neither new nor difficult to grasp. From an
>     operators perspective, I don't believe it adds tons of complexity. It
>     does require the operator to deploy a replicated storage environment but
>     I believe all services require that.
> 
>     Cheers,
>     Flavio
> 
>     P.S: Sorry for my late answer or lack of it. I lost *all* my emails
>     yesterday and I'm working on recovering them.
> 
>     [0] https://bugs.launchpad.net/zaqar/+bug/1372335
> 
>     --
>     @flaper87
>     Flavio Percoco
> 
>     _______________________________________________
>     OpenStack-dev mailing list
>     OpenStack-dev at lists.openstack.org
>     <mailto:OpenStack-dev at lists.openstack.org>
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 


-- 
@flaper87
Flavio Percoco