[openstack-dev] [Marconi] Kafka support and high throughput

Hochmuth, Roland M roland.hochmuth at hp.com
Sun Jun 1 15:01:40 UTC 2014


There are some folks in HP evaluating different messaging technologies for
Marconi, such as RabbitMQ and Kafka. I'll ping them and maybe they can
share
some information.

On a related note, the Monitoring as a Service solution we are working
on uses Kafka. This was just open-sourced at,
https://github.com/hpcloud-mon,
and will be moving over to StackForge starting next week. The architecture
is at,
https://github.com/hpcloud-mon/mon-arch.

I haven't really looked at Marconi. If you are interested in
throughput, low latency, durability, scale and fault-tolerance Kafka
seems like a great choice.

It has been also pointed out from various sources that possibly Kafka
could be another oslo.messaging transport. Are you looking into that as
that would be very interesting to me and something that is on my task
list that I haven't gotten to yet.


On 5/30/14, 7:03 AM, "Keith Newstadt" <keith_newstadt at symantec.com> wrote:

>Has anyone given thought to using Kafka to back Marconi?  And has there
>been discussion about adding high throughput APIs to Marconi.
>
>We're looking at providing Kafka as a messaging service for our
>customers, in a scenario where throughput is a priority.  We've had good
>luck using both streaming HTTP interfaces and long poll interfaces to get
>high throughput for other web services we've built.  Would this use case
>be appropriate in the context of the Marconi roadmap?
>
>Thanks,
>Keith Newstadt
>keith_newstadt at symantec.com
>




More information about the OpenStack-dev mailing list