Thanks for your help Erik. All is fine with NTP. Exactly the same result with " docker exec -it rabbitmq rabbitmqctl cluster_status" on the 2 nodes.
I Will try this:
On both controllers do: docker rm rabbitmq docker volume rm rabbitmq
Then kolla-ansible --tags rabbitmq deploy
Franck VEDEL Dép. Réseaux Informatiques & Télécoms IUT1 - Univ GRENOBLE Alpes 0476824462 Stages, Alternance, Emploi.
Le 11 nov. 2022 à 16:59, Erik McCormick emccormick@cirrusseven.com a écrit :
On Fri, Nov 11, 2022 at 3:05 AM Franck VEDEL <franck.vedel@univ-grenoble-alpes.fr mailto:franck.vedel@univ-grenoble-alpes.fr> wrote:
Thanks for your help, really. My cluster: 2 controllers nodes, OVS, L3-HA. All nodes had to be rebooted All is working for example with external networks (so dhcp on external networks). There are no dead containers, all seems ok.
I try to create a new instance on a L3 network. No ERROR in neutron*.log. The only error is nova-api.log:
Example: 2022-11-11 08:45:54.452 42 ERROR oslo.messaging._drivers.impl_rabbit [-] [8b6fd776-f096-4c8a-927e-88225a3adb43] AMQP server on 10.0.5.109:5672 http://10.0.5.109:5672/ is unreachable: <RecoverableConnectionError: unknown error>. Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: <RecoverableConnectionError: unknown error>
But on the first node (10.0.5.109 on the internal network) « netstat -atnp |wc-l » ———>>> 505 connections
Sounds to me like Rabbit is broken. This could also be an issue with NTP which I asked about earlier. Did you confirm your systems are all correctly synced to the same time source?
You can check the status of rabbit on each control node with:
docker exec -it rabbitmq rabbitmqctl cluster_status
Output should show the same on both of your controllers. If not, restart your rabbit containers. If they won't come back properly, you could destroy and redeploy just those two containers l
On both controllers do: docker rm rabbitmq docker volume rm rabbitmq
Then kolla-ansible --tags rabbitmq deploy
So…. if I backup /etc/kolla, my glance images, my configuration files… if a do « koll-ansible destroy », is next step « kolla-ansible bootstraps…. » and preaches, and deploy, or directly deploy ?
What’s the difference with cleanup-containers ?
You don't need to bootstrap again. That just installs prerequisites which won't get removed from the destroy. Just go right to doing kolla-ansible deploy again. Do remember this will give you a brand new Openstack with nothing preserved from before.
cleanup-containers alone may leave behind some docker tweaks that Neutron needs. It probably doesn't matter if you're going to just redeploy the same configuration though so go ahead and use that instead.
-Erik