[ops] Automatically recover guests from down host
All; What is the most effective means of having the OpenStack cluster restart guests when a hypervisor host fails? We're running OpenStack Victoria, installed manually through packages. My apologies, but my Google foo fails me on this issue; I don't know how to ask it the question. I recognize that OpenStack covers a great many different deployment scenarios, and in many of these this isn't feasible. In our case, images, volumes, and ephemeral storage are all on our Ceph cluster, so all storage is always available to all hypervisor hosts. I also recognize that resource restrictions mean that even in an environment such as mine, not all failed guests may be able to be restarted on new hosts. I'm ok with a dumb best effort, at least for now. Is there something already present in OpenStack which would allow this? Thank you, Dominic L. Hilsbos, MBA Vice President - Information Technology Perform Air International Inc. DHilsbos@PerformAir.com www.PerformAir.com
Hi, There is an option in nova to evacuate host. Triggering this will rebuild all the vms running on failed host to be scheduled on other host or reserved host. You can also try Openstack Masakri that is the instance HA service for openstack. Ammad On Fri, Jun 11, 2021 at 9:28 PM <DHilsbos@performair.com> wrote:
All;
What is the most effective means of having the OpenStack cluster restart guests when a hypervisor host fails? We're running OpenStack Victoria, installed manually through packages.
My apologies, but my Google foo fails me on this issue; I don't know how to ask it the question.
I recognize that OpenStack covers a great many different deployment scenarios, and in many of these this isn't feasible. In our case, images, volumes, and ephemeral storage are all on our Ceph cluster, so all storage is always available to all hypervisor hosts.
I also recognize that resource restrictions mean that even in an environment such as mine, not all failed guests may be able to be restarted on new hosts. I'm ok with a dumb best effort, at least for now.
Is there something already present in OpenStack which would allow this?
Thank you,
Dominic L. Hilsbos, MBA Vice President - Information Technology Perform Air International Inc. DHilsbos@PerformAir.com www.PerformAir.com
-- Regards,
Syed Ammad Ali
On Fri, Jun 11, 2021 at 8:13 PM Ammad Syed <syedammad83@gmail.com> wrote:
Hi,
There is an option in nova to evacuate host. Triggering this will rebuild all the vms running on failed host to be scheduled on other host or reserved host.
You can also try Openstack Masakri that is the instance HA service for openstack.
Just following up on this: the project is called Masakari and has docs in: https://docs.openstack.org/masakari/latest/ The team can be reached on OFTC ( https://www.oftc.net/ ) at #openstack-masakari or via this mailing list with [masakari] tag in subject.
Ammad
On Fri, Jun 11, 2021 at 9:28 PM <DHilsbos@performair.com> wrote:
All;
What is the most effective means of having the OpenStack cluster restart guests when a hypervisor host fails? We're running OpenStack Victoria, installed manually through packages.
My apologies, but my Google foo fails me on this issue; I don't know how to ask it the question.
I recognize that OpenStack covers a great many different deployment scenarios, and in many of these this isn't feasible. In our case, images, volumes, and ephemeral storage are all on our Ceph cluster, so all storage is always available to all hypervisor hosts.
I also recognize that resource restrictions mean that even in an environment such as mine, not all failed guests may be able to be restarted on new hosts. I'm ok with a dumb best effort, at least for now.
Is there something already present in OpenStack which would allow this?
One of the goals Masakari has is to introduce a system of recovery prioritisation to go beyond the "dumb best effort" mentioned. For now it's pretty simple but matches your requirements. -yoctozepto
participants (3)
-
Ammad Syed
-
DHilsbos＠performair.com
-
Radosław Piliszek