Open Stack

On 09/09/2014 07:04 PM, Samuel Merritt wrote:
> On 9/9/14, 4:47 PM, Devananda van der Veen wrote:
>> On Tue, Sep 9, 2014 at 4:12 PM, Samuel Merritt <sam at swiftstack.com>
>> wrote:
>>> On 9/9/14, 12:03 PM, Monty Taylor wrote:
>> [snip]
>>>> So which is it? Because it sounds like to me it's a thing that actually
>>>> does NOT need to diverge in technology in any way, but that I've been
>>>> told that it needs to diverge because it's delivering a different
>>>> set of
>>>> features - and I'm pretty sure if it _is_ the thing that needs to
>>>> diverge in technology because of its feature set, then it's a thing I
>>>> don't think we should be implementing in python in OpenStack because it
>>>> already exists and it's called AMQP.
>>>
>>>
>>> Whether Zaqar is more like AMQP or more like email is a really strange
>>> metric to use for considering its inclusion.
>>>
>>
>> I don't find this strange at all -- I had been judging the technical
>> merits of Zaqar (ex-Marconi) for the last ~18 months based on the
>> understanding that it aimed to provide Queueing-as-a-Service, and
>> found its delivery of that to be lacking on technical grounds. The
>> implementation did not meet my view of what a queue service should
>> provide; it is based on some serious antipatterns (storing a queue in
>> an RDBMS is probably the most obvious); and in fact, it isn't even
>> queue-like in the access patterns enabled by the REST API (random
>> access to a set != a queue). That was the basis for a large part of my
>> objections to the project over time, and a source of frustration for
>> me as the developers justified many of their positions rather than
>> accepted feedback and changed course during the incubation period. The
>> reason for this seems clear now...
>>
>> As was pointed out in the TC meeting today, Zaqar is (was?) actually
>> aiming to provide Messaging-as-a-Service -- not queueing as a service!
>> This is another way of saying "it's more like email and less like
>> AMQP", which means my but-its-not-a-queue objection to the project's
>> graduation is irrelevant, and I need to rethink about all my previous
>> assessments of the project.
>>
>> The questions now before us are:
>> - should OpenStack include, in the integrated release, a
>> messaging-as-a-service component?
>
> I certainly think so. I've worked on a few reasonable-scale web
> applications, and they all followed the same pattern: HTTP app servers
> serving requests quickly, background workers for long-running tasks, and
> some sort of durable message-broker/queue-server thing for conveying
> work from the first to the second.
>
> A quick straw poll of my nearby coworkers shows that every non-trivial
> web application that they've worked on in the last decade follows the
> same pattern.
>
> While not *every* application needs such a thing, web apps are quite
> common these days, and Zaqar satisfies one of their big requirements.
> Not only that, it does so in a way that requires much less babysitting
> than run-your-own-broker does.

Right. But here's the thing.

What you just described is what we all thought zaqar was aiming to be in 
the beginning. We did not think it was a GOOD implementation of that, so 
while we agreed that it would be useful to have one of those, we were 
not crazy about the implementation.

But that is what we have recently learned that zaqar is NOT. It is not a 
message queue in the way that you describe. It is something else. Now, I 
tend to believe that that something else is also potentially interesting 
- but the question of whether that thing is a thing that we want, and 
whether _this_ implementation of that this is a good implementation are 
the questions at hand.


>> - is Zaqar a technically sound implementation of such a service?
>>
>> As an aside, there are still references to Zaqar as a queue in both
>> the wiki [0], in the governance repo [1], and on launchpad [2].
>>
>> Regards,
>> Devananda
>>
>>
>> [0] "Multi-tenant queues based on Keystone project IDs"
>>    https://wiki.openstack.org/wiki/Zaqar#Key_features
>>
>> [1] "Queue service" is even the official OpenStack Program name, and
>> the mission statement starts with "To produce an OpenStack message
>> queueing API and service."
>>
>> http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml#n315
>>
>>
>> [2] "Zaqar is a new OpenStack project to create a multi-tenant cloud
>> queuing service...."
>>    https://launchpad.net/zaqar
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>