Right, thank you - I thought and hoped that was the intent. I'm just doing: docker stop mariadb docker start mariadb I've just done the same with host 3, and while stopped grastate.dat changed to `safe_to_bootstrap: 1` on host 2, and seqno: appeared in host 3's grastate.dat. Starting it again worked fine. docker inspect mariadb differs on the hosts, 1 has: "Env": [ ... "BOOTSTRAP_ARGS=--wsrep-new-cluster", while the other two don't. /var/log/kolla/mariadb/mariadb.log https://termbin.com/px15 with safe_to_bootstrap: 0, container stops again https://termbin.com/meli after removing /var/lib/docker/volumes/mariadb/_data/<all but mysql/>, separate cluster results I've just tested a dev cluster with the same steps, and it doesn't have the same problem - no Env entry, no problems restarting. I'm now wondering if it relates to the database migration and restore we did a while back, coming from another cluster with rsync replication and switching over to mariabackup. Next, I deleted the container in the dev cluster, and ran mariadb_recovery play - worked fine. I was hesitant to do this in the test cluster I'm working on because of other settings we have handcrafted for now, but after a few trial runs, I'm more confident. ... Yep, that fixed it - docker stop/start mariadb now works. BOOTSTRAP_ARGS is gone from the Env in the container too. Thank you for giving me the confidence to prod it further. Greg. On 9/6/22 16:53, Mark Goddard wrote:

Hi Greg,

It should only have that option set when bootstrapping the cluster. How are you stopping and starting the container? Could you provide logs?

Mark

On Thu, 9 Jun 2022 at 09:17, Gregory Orange <gregory.orange@pawsey.org.au <mailto:gregory.orange@pawsey.org.au>> wrote:

Hi all,

I wanted to temporarily change a mariadb config in /etc/kolla/mariadb/galera.cnf so I intended to stop mariadb on host 1, make the change, start it again, then repeat in turn for hosts 2 and 3.

Upon starting it on host 1, it doesn't rejoin, but creates its own separate cluster. Dutiful really, since logs show:

WSREP: 'wsrep-new-cluster' option used, bootstrapping the cluster

How do I avoid that? General advice on handling this kind of activity would also be welcome.

Thank you, Greg.

-- Gregory Orange Cloud System Administrator Scientific Platforms Team building representative Pawsey Supercomputing Centre, CSIRO