Open Stack

On 10/06/14 09:48 +0100, Mark McLoughlin wrote:
>On Mon, 2014-06-09 at 19:31 +0000, Kurt Griffiths wrote:
>> Lately we have been talking about writing drivers for traditional
>> message brokers that will not be able to support the message feeds
>> part of the API. I’ve started to think that having a huge part of the
>> API that may or may not “work”, depending on how Marconi is deployed,
>> is not a good story for users, esp. in light of the push to make
>> different clouds more interoperable.
>
>Perhaps the first point to get super clear on is why drivers for
>traditional message brokers are needed. What problems would such drivers
>address? Who would the drivers help? Would the Marconi team recommend
>using any of those drivers for a production queuing service? Would the
>subset of Marconi's API which is implementable by these drivers really
>be useful for application developers?
>

These are all very good questions that should be taken into account
when evaluating not just the drivers under discussion but also future
drivers.

As mentioned in my previous email on this thread, I don't think we are
ready to make a final/good decision here because it's still not clear
what the trade-off is. Some things are clear, though:

1. Marconi relies on a store-forward message delivery model
2. As of now (v1.x) it relies on Queues as a first-citizen resource in
the API.

Ideally, those drivers would be used when a higher throughput is
needed since they're known to be faster and created to solve this
issue.

There's something else that Marconi brings to those technologies,
which is the ability to create clusters of storage - as of now those
storage would be segmented in a pre-configured, per-queue basis. In
other words, Marconi has support for storage shards. This helps
solving some of the scaling issues in some of the existing queuing
technologies. This is probably not really relevant but worth
mentioning.

>I'd like to understand that in more detail because I worry the Marconi
>team is being pushed into adding these drivers without truly believing
>they will be useful. And if that would not be a sane context to make a
>serious architectural change.
>
>OTOH if there are real, valid use cases for these drivers, then
>understanding those would inform the architecture decision.

Completely agreed with the feeling.

I'm one of those willing to support AMQP brokers but, FWIW, not
blindly. For now, the AMQP driver is a "research driver" that should
help answering some of the questions you asked above. The work on that
driver is happening in a separate repo.

Your questions, as mentioned above, are a good starting point for this
discussion.

Thanks for the great feedback.

Flavio

-- 
@flaper87
Flavio Percoco
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140610/3977dab3/attachment.pgp>