[openstack-dev] [Marconi] Kafka support and high throughput

Janczuk, Tomasz tomasz.janczuk at hp.com
Mon Jun 2 18:46:29 UTC 2014


Have you put HTTP protocol in front of Kafka, or are you using Kafka¹s
native protocol?

Can you also expand a little on your performance requirements? What does
³high throughput² mean to you in terms of the messaging patterns (# of
producers and consumers, # of queues and queue partitions, message size,
desired throughput)?

Tomasz Janczuk

On 5/31/14, 11:56 PM, "Flavio Percoco" <flavio at redhat.com> wrote:

>On 30/05/14 06:03 -0700, Keith Newstadt wrote:
>>Has anyone given thought to using Kafka to back Marconi?  And has there
>>been discussion about adding high throughput APIs to Marconi.
>>We're looking at providing Kafka as a messaging service for our
>>customers, in a scenario where throughput is a priority.  We've had good
>>luck using both streaming HTTP interfaces and long poll interfaces to
>>get high throughput for other web services we've built.  Would this use
>>case be appropriate in the context of the Marconi roadmap?
>Kafka would be a good store to back Marconi with. We've had some
>feedback from the community w.r.t Kafka and there seems to be lot of
>interest on it. The team is not currently targeting it but we could
>probably do something after the J release.
>That said, Marconi's plugin architecture allows people to create
>third-party drivers and obviously use them. It'd be really nice to see
>some work going on that area as an external driver.
>Flavio Percoco
>OpenStack-dev mailing list
>OpenStack-dev at lists.openstack.org

More information about the OpenStack-dev mailing list