Ok Jay,

let me to describe my environment.

I have an openstack made up of 3 controllers nodes ad several compute nodes.

The controller nodes services are controlled by pacemaker and the compute nodes services are controlled by remote pacemaker.

My hardware is Dell so I am using ipmi fencing device .

I wrote a service controlled by pacemaker:

this service controls if a compude node fails and for avoiding split brains if a compute node does nod respond on the management network and on storage network the stonith poweroff the node and then execute a nova host-evacuate.

Anycase to have a simulation before writing the service I described above you can do as follows:

connect on one compute node where some virtual machines are running

run the command: echo 'c' > /proc/sysrq-trigger (it stops immediately the node like in case of failure)

On a controller node run: nova host-evacuate "name of failed compute node"

Instances running on the failed compute node should be restarted on another compute node

Ignazio

Il giorno gio 11 lug 2019 alle ore 11:57 Jay See <jayachander.it@gmail.com> ha scritto:

Hi ,

I have tried on a failed compute node which is in power off state now.
I have tried on a running compute node, no errors. But nothing happens.
On running compute node - Disabled the compute service and tried migration also.

May be I might have not followed proper steps. Just wanted to know the steps you have followed. Otherwise, I was planning to manual migration also if possible.
~Jay.

On Thu, Jul 11, 2019 at 11:52 AM Ignazio Cassano <ignaziocassano@gmail.com> wrote:
Hi Jay,
would you like to evacuate a failed compute node or evacuate a running compute node ?

Ignazio

Il giorno gio 11 lug 2019 alle ore 11:48 Jay See <jayachander.it@gmail.com> ha scritto:
Hi Ignazio,

I am trying to evacuate the compute host on older version (mitaka).
Could please share the process you followed. I am not able to succeed with openstack live-migration fails with error message (this is known issue in older versions) and nova live-ligration - nothing happens even after initiating VM migration. It is almost 4 days.

~Jay.

On Thu, Jul 11, 2019 at 11:31 AM Ignazio Cassano <ignaziocassano@gmail.com> wrote:
I am sorry.
For simulating an host crash I used a wrong procedure.
Using "echo 'c' > /proc/sysrq-trigger" all work fine

Il giorno gio 11 lug 2019 alle ore 11:01 Ignazio Cassano <ignaziocassano@gmail.com> ha scritto:
Hello All,
on ocata when I poweroff a node with active instance , doing a nova host-evacuate works fine
and instances are restartd on an active node.
On queens it does non evacuate instances but nova-api reports for each instance the following:

2019-07-11 10:19:54.745 13811 INFO nova.api.openstack.wsgi [req-daad0a7d-87ce-41bf-b096-a70fc306db5c 0c7a2d6006614fe2b3e81e47377dd2a9 c26f8d35f85547c4add392a221af1aab - default default] HTTP exception thrown: Cannot 'evacuate' instance e8485a5e-3623-4184-bcce-cafd56fa60b3 while it is in task_state powering-off

So it poweroff all instance on the failed node but does not start them on active nodes

What is changed ?
Ignazio

--

P SAVE PAPER – Please do not print this e-mail unless absolutely necessary.

--

P SAVE PAPER – Please do not print this e-mail unless absolutely necessary.