Open Stack

Yes, we're doing this when all MariaDB containers are down.
But we still curious about this problem. Ordering to shutdown then boot up
the MariaDB cluster still can caused this issue.

My initial guess is that the docker is startup earlier than network, caused
they can't connect each other.
But it still a guess because the connection usually back if restart
container manually. At least this is what we solve if the service can't
connect to DB or AMQP but both them are fine.

Perhaps I may try this. For now, it seems like using mariadb_recovery is
the only way to let MariaDB back online if reboot the whole cluster right?

Mark Goddard <mark at stackhpc.com> 於 2019年11月15日 週五 下午8:07寫道：

> On Fri, 15 Nov 2019 at 03:28, Eddie Yen <missile0407 at gmail.com> wrote:
> >
> > Hi everyone,
> > I want to ask about the order of shutdown MariaDB (or mean controller)
> node.
> >
> > For previous steps we found is usually shutdown slaves first, then
> master [1].
> > But we found that the MariaDB still get container restarting issue even
> I followed the step after booting up the cluster.
> >
> > Below is that I did when shutdown/boot up controller.
> > 1. Shutdown the slaves first, then master
> > 2. Boot master first, then slaves.
> >
> > For looking which one is master, we usually looking for the haproxy log
> and find which mariadb node that the last session access the DB.
> > Or looking for which mariadb container has "--wsrep-new-cluster" in
> BOOTSTRAP_ARGS.
> >
> > Does anyone has experience about this?
>
> Hi Eddie,
> You can use the kolla-ansible mariadb_recovery command to bootstrap a
> cluster where all nodes have gone down.
> Mark
> >
> > Many thanks,
> > Eddie.
> >
> > [1] https://bugs.launchpad.net/kolla-ansible/+bug/1712087
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20191115/928f2476/attachment.html>