[kolla] restart mariadb container on host 1
gregory.orange at pawsey.org.au
Thu Jun 9 11:45:20 UTC 2022
Right, thank you - I thought and hoped that was the intent.
I'm just doing:
docker stop mariadb
docker start mariadb
I've just done the same with host 3, and while stopped grastate.dat
changed to `safe_to_bootstrap: 1` on host 2, and seqno: appeared in host
3's grastate.dat. Starting it again worked fine.
docker inspect mariadb differs on the hosts, 1 has:
"Env": [ ... "BOOTSTRAP_ARGS=--wsrep-new-cluster",
while the other two don't.
https://termbin.com/px15 with safe_to_bootstrap: 0, container stops again
https://termbin.com/meli after removing
/var/lib/docker/volumes/mariadb/_data/<all but mysql/>, separate cluster
I've just tested a dev cluster with the same steps, and it doesn't have
the same problem - no Env entry, no problems restarting.
I'm now wondering if it relates to the database migration and restore we
did a while back, coming from another cluster with rsync replication and
switching over to mariabackup.
Next, I deleted the container in the dev cluster, and ran
mariadb_recovery play - worked fine. I was hesitant to do this in the
test cluster I'm working on because of other settings we have
handcrafted for now, but after a few trial runs, I'm more confident.
Yep, that fixed it - docker stop/start mariadb now works. BOOTSTRAP_ARGS
is gone from the Env in the container too.
Thank you for giving me the confidence to prod it further.
On 9/6/22 16:53, Mark Goddard wrote:
> Hi Greg,
> It should only have that option set when bootstrapping the cluster. How
> are you stopping and starting the container? Could you provide logs?
> On Thu, 9 Jun 2022 at 09:17, Gregory Orange
> <gregory.orange at pawsey.org.au <mailto:gregory.orange at pawsey.org.au>> wrote:
> Hi all,
> I wanted to temporarily change a mariadb config in
> /etc/kolla/mariadb/galera.cnf so I intended to stop mariadb on host 1,
> make the change, start it again, then repeat in turn for hosts 2 and 3.
> Upon starting it on host 1, it doesn't rejoin, but creates its own
> separate cluster. Dutiful really, since logs show:
> WSREP: 'wsrep-new-cluster' option used, bootstrapping the cluster
> How do I avoid that? General advice on handling this kind of activity
> would also be welcome.
> Thank you,
Cloud System Administrator
Scientific Platforms Team building representative
Pawsey Supercomputing Centre, CSIRO
More information about the openstack-discuss