Open Stack

We’re running kolla-ansible Train. I followed the recommendations in [1] and ended up with the following config:

definitions.json (in the rabbitmq container):

{
  "vhosts": [
    {"name": "/"}  ],
  "users": [
    {"name": "openstack", "password": "<password>", "tags": "administrator"}  ],
  "permissions": [
    {"user": "openstack", "vhost": "/", "configure": ".*", "write": ".*", "read": ".*"}  ],
  "policies":[
    {"vhost": "/", "name": "notifications-expire", "pattern": "^(notifications_designate|versioned_notifications).*", "apply-to": "queues", "definition": {"message-ttl":600000,"expires":1200000}, "priority":0},
    {"vhost": "/", "name": "ha-all", "pattern": "^(?!(amq\.)|(.*_fanout_)|(reply_)).*", "apply-to": "all", "definition": {"ha-mode":"all","ha-promote-on-shutdown": "always", "ha-sync-mode":"automatic"}, "priority":0}  ]
}

etc/kolla/config/global.conf:

[oslo_messaging_rabbit]
amqp_durable_queues = True

This fixed some issues, but we seem to have a new issue so I must be missing a setting. When we stop the RMQ container on ctrl1, designate stops working (DNS records are not created nor deleted) and I see this in designate-sink.log:

2022-03-17 19:21:29.261 28 ERROR oslo.messaging._drivers.impl_rabbit [req-2c0cd9f4-5331-4697-9c3e-eece475a52af - - - - -] Failed to consume message from queue: Queue.declare: (404) NOT_FOUND - home node 'rabbit at de6-ctrl1' of durable queue 'notifications_designate.info' in vhost '/' is down or inaccessible: amqp.exceptions.NotFound: Queue.declare: (404) NOT_FOUND - home node 'rabbit at de6-ctrl1' of durable queue 'notifications_designate.info' in vhost '/' is down or inaccessible

When I look at the notifications_designate.info queue in the web interface, it appears to have moved to ctrl2:

Features             
durable:               true
Policy    notifications-expire
Operator policy                
Effective policy definition            
expires: 1200000
message-ttl:       600000
Node     rabbit at qde3-ctrl2

When I look at designate.conf in the designate_sink containers I don’t see anything pointing only to ctrl1:

transport_url = rabbit://openstack:<pwd>@<ctrl1>:5672,openstack:<pwd>@<ctrl2>:5672,openstack:<pwd>@<ctrl3>:5672//

But it appears that Designate still tries to use the queue on ctrl1. After I bring up ctrl1, the notifications_designate.info queue remains on ctrl2, but Designate starts working.

What am I missing?

[1] https://wiki.openstack.org/wiki/Large_Scale_Configuration_Rabbit