Open Stack

On 9/9/14, 4:47 PM, Devananda van der Veen wrote:
> On Tue, Sep 9, 2014 at 4:12 PM, Samuel Merritt <sam at swiftstack.com> wrote:
>> On 9/9/14, 12:03 PM, Monty Taylor wrote:
> [snip]
>>> So which is it? Because it sounds like to me it's a thing that actually
>>> does NOT need to diverge in technology in any way, but that I've been
>>> told that it needs to diverge because it's delivering a different set of
>>> features - and I'm pretty sure if it _is_ the thing that needs to
>>> diverge in technology because of its feature set, then it's a thing I
>>> don't think we should be implementing in python in OpenStack because it
>>> already exists and it's called AMQP.
>>
>>
>> Whether Zaqar is more like AMQP or more like email is a really strange
>> metric to use for considering its inclusion.
>>
>
> I don't find this strange at all -- I had been judging the technical
> merits of Zaqar (ex-Marconi) for the last ~18 months based on the
> understanding that it aimed to provide Queueing-as-a-Service, and
> found its delivery of that to be lacking on technical grounds. The
> implementation did not meet my view of what a queue service should
> provide; it is based on some serious antipatterns (storing a queue in
> an RDBMS is probably the most obvious); and in fact, it isn't even
> queue-like in the access patterns enabled by the REST API (random
> access to a set != a queue). That was the basis for a large part of my
> objections to the project over time, and a source of frustration for
> me as the developers justified many of their positions rather than
> accepted feedback and changed course during the incubation period. The
> reason for this seems clear now...
>
> As was pointed out in the TC meeting today, Zaqar is (was?) actually
> aiming to provide Messaging-as-a-Service -- not queueing as a service!
> This is another way of saying "it's more like email and less like
> AMQP", which means my but-its-not-a-queue objection to the project's
> graduation is irrelevant, and I need to rethink about all my previous
> assessments of the project.
>
> The questions now before us are:
> - should OpenStack include, in the integrated release, a
> messaging-as-a-service component?

I certainly think so. I've worked on a few reasonable-scale web 
applications, and they all followed the same pattern: HTTP app servers 
serving requests quickly, background workers for long-running tasks, and 
some sort of durable message-broker/queue-server thing for conveying 
work from the first to the second.

A quick straw poll of my nearby coworkers shows that every non-trivial 
web application that they've worked on in the last decade follows the 
same pattern.

While not *every* application needs such a thing, web apps are quite 
common these days, and Zaqar satisfies one of their big requirements. 
Not only that, it does so in a way that requires much less babysitting 
than run-your-own-broker does.

> - is Zaqar a technically sound implementation of such a service?
>
> As an aside, there are still references to Zaqar as a queue in both
> the wiki [0], in the governance repo [1], and on launchpad [2].
>
> Regards,
> Devananda
>
>
> [0] "Multi-tenant queues based on Keystone project IDs"
>    https://wiki.openstack.org/wiki/Zaqar#Key_features
>
> [1] "Queue service" is even the official OpenStack Program name, and
> the mission statement starts with "To produce an OpenStack message
> queueing API and service."
>    http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml#n315
>
> [2] "Zaqar is a new OpenStack project to create a multi-tenant cloud
> queuing service...."
>    https://launchpad.net/zaqar