[ops] [kolla] RabbitMQ High Availability
hberaud at redhat.com
Thu Dec 9 07:45:10 UTC 2021
Le mer. 8 déc. 2021 à 11:48, Bogdan Dobrelya <bdobreli at redhat.com> a écrit :
> Please see inline
> >> I read this with great interest because we are seeing this issue.
> >> 1. We are running kola-ansible Train, and our RMQ version is 3.7.23.
> Should we be upgrading our Train clusters to use 3.8.x?
> >> 2. Document  recommends policy
> '^(?!(amq\.)|(.*_fanout_)|(reply_)).*'. I don't see this in our ansible
> playbooks, nor in any of the config files in the RMQ container. What would
> this look like in Ansible, and what should the resulting container config
> look like?
> >> 3. It appears that we are not setting "amqp_durable_queues = True".
> What does this setting look like in Ansible, and what file does it go into?
> > Note that even having rabbit HA policies adjusted like that and its HA
> > replication factor  decreased (e.g. to a 2), there still might be
> > high churn caused by a large enough number of replicated durable RPC
> > topic queues. And that might cripple the cloud down with the incurred
> > I/O overhead because a durable queue requires all messages in it to be
> > persisted to a disk (for all the messaging cluster replicas) before they
> > are ack'ed by the broker.
> > Given that said, Oslo messaging would likely require a more granular
> > control for topic exchanges and the durable queues flag - to tell it to
> > declare as durable only the most critical paths of a service. A single
> > config setting and a single control exchange per a service might be not
> > enough.
> Also note that therefore, amqp_durable_queue=True requires dedicated
> control exchanges configured for each service. Those that use
> 'openstack' as a default cannot turn the feature ON. Changing it to a
> service specific might also cause upgrade impact, as described in the
> topic .
The same is true for `amqp_auto_delete=True`. That requires dedicated
control exchanges else it won't work if each service defines its own policy
on a shared control exchange (e.g `openstack`) and if policies differ from
>  https://review.opendev.org/q/topic:scope-config-opts
> > There are also race conditions with durable queues enabled, like . A
> > solution could be where each service declare its own dedicated control
> > exchange with its own configuration.
> > Finally, openstack components should add perhaps a *.next CI job to test
> > it with durable queues, like 
> >  https://www.rabbitmq.com/ha.html#replication-factor
> > 
> >  https://review.opendev.org/c/openstack/nova/+/820523
> >> Does anyone have a sample set of RMQ config files that they can share?
> >> It looks like my Outlook has ruined the link; reposting:
> >>  https://wiki.openstack.org/wiki/Large_Scale_Configuration_Rabbit
> > --
> > Best regards,
> > Bogdan Dobrelya,
> > Irc #bogdando
> Best regards,
> Bogdan Dobrelya,
> Irc #bogdando
Senior Software Engineer at Red Hat
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the openstack-discuss