[Openstack] [openstack-dev] High Available queues in rabbitmq

Rosa, Andrea (HP Cloud Services) andrea.rosa at hp.com
Fri Jul 27 14:05:28 UTC 2012


Hi 

>As for consumer cancellation notifications - I need to remember when
>exactly they happen in an HA setup. Maybe you're right.

With the HA active/active configuration clients always consume from the Master node so we can have a scenario where the queue is declared on a Slave node but the clients are consuming from the Master node.
In this situation if the master node goes down the client doesn't get any error/exception from the connection and the consumer is abruptly disconnected, so the client needs to re-consume the queue (register a new consumer) to be able to get messages from the queue.
The consumer cancellation notifications is a way for a client to get a notification (via a basic.cancel message) when the consumer has been disconnected.

With the HA configuration we should care about of two things:
1 connection dying : this is already addressed by the existing code
2 basic.cancel notification received by the clients: in my opinion this is not yet addressed

>As for duplicated messages - the situation here is no different
>whether you have 1 or many rabbits; reconnection logic was already in
>place. Perhaps something should be done here, but it seems that the
>lack of this logic didn't hurt anyone so far. Maybe it is because IIRC
>messages are ack'd immediately and the failure window is very small.

Yes you are absolutely right, but with the HA configuration is more likely to have retransmissions.
If I have a single rabbit and NOT durable queue, all messages (also messages not yet ack'ed) in the queues are lost if the node goes down, that means that when the server restarts there are no messages left and so no retransmissions.
With multiple rabbits, messages not yet ack'ed are mirrored in the other queues and in the event of failure of the master those messages will be retransmitted.


>> Is there some plan to have a blueprint for this change?
>I don't have such a plan. Should I?

I need to deeply investigate and tests the HA Active/Active configuration, but If I can recreate some "complex" configuration to prove that we need to deal with the basic.cancel messages maybe it's worth to have one.

Regards
--
Andrea Rosa





More information about the Openstack mailing list