Hi! It´s a lot of information. From engine: 2021-12-06 20:02:33.502 7 INFO masakari.engine.manager [req-034e0a9e-1b34-4db3-a2e0-6d61a369ba6c 6c128366b66346d38fb5493adf0cf666 e39a7ea7d17046b5b97b2253bf8195bc - - -] Processing notification 2468563d-70c5-41ef-ad04-a51b4ad3dd4d of type: COMPUTE_HOST 2021-12-06 20:02:34.052 7 INFO masakari.compute.nova [req-882d4151-0549-4cb2-acbb-90509073179c nova - - - -] Disable nova-compute on ctl02-hml.amt.net.br 2021-12-06 20:02:34.113 7 INFO masakari.engine.drivers.taskflow.host_failure [req-882d4151-0549-4cb2-acbb-90509073179c nova - - - -] Sleeping 180 sec before starting recovery thread until nova recognizes the node down. 2021-12-06 20:05:34.130 7 INFO masakari.compute.nova [req-c570bdef-0786-47bd-94fc-e6c1396eabb1 nova - - - -] Fetch Server list on ctl02-hml.amt.net.br 2021-12-06 20:05:35.166 7 INFO masakari.compute.nova [req-adec6e54-15ff-4a28-98c3-01070ba2cf87 nova - - - -] Call get server command for instance 618e44e8-248f-4f50-a760-581972352af8 2021-12-06 20:05:35.949 7 INFO masakari.compute.nova [req-49414eed-fdcf-4fd0-b803-e5cc5f2f9227 nova - - - -] Call get server command for instance 746178b2-14ce-4ce2-83f6-2bf9d613a887 2021-12-06 20:05:35.955 7 INFO masakari.compute.nova [req-e548525d-74f7-42c4-8045-2f63d645f76f nova - - - -] Call get server command for instance 618e44e8-248f-4f50-a760-581972352af8 2021-12-06 20:05:36.739 7 INFO masakari.compute.nova [req-9cea2ff2-0d32-42a1-8605-b889d95cadc0 nova - - - -] Call lock server command for instance 618e44e8-248f-4f50-a760-581972352af8 2021-12-06 20:05:36.747 7 INFO masakari.compute.nova [req-bc3bdea3-e923-4c08-bf84-bea711465dce nova - - - -] Call get server command for instance 746178b2-14ce-4ce2-83f6-2bf9d613a887 2021-12-06 20:05:37.391 7 INFO masakari.compute.nova [req-9cea2ff2-0d32-42a1-8605-b889d95cadc0 nova - - - -] Call evacuate command for instance 618e44e8-248f-4f50-a760-581972352af8 on host None 2021-12-06 20:05:37.478 7 INFO masakari.compute.nova [req-84d38844-ae87-4767-b822-0099bc40aa16 nova - - - -] Call lock server command for instance 746178b2-14ce-4ce2-83f6-2bf9d613a887 2021-12-06 20:05:38.059 7 INFO masakari.compute.nova [req-d57bd685-2e42-48e0-bf7d-9978ac516451 nova - - - -] Call get server command for instance 618e44e8-248f-4f50-a760-581972352af8 2021-12-06 20:05:38.102 7 INFO masakari.compute.nova [req-84d38844-ae87-4767-b822-0099bc40aa16 nova - - - -] Call evacuate command for instance 746178b2-14ce-4ce2-83f6-2bf9d613a887 on host None 2021-12-06 20:05:38.734 7 INFO masakari.compute.nova [req-25b4bb9c-2e05-409a-bae8-fd7c8348ba5b nova - - - -] Call get server command for instance 746178b2-14ce-4ce2-83f6-2bf9d613a887 2021-12-06 20:05:39.062 7 INFO masakari.compute.nova [req-3d69363b-aa6c-4752-be52-285bc36f25c2 nova - - - -] Call get server command for instance 618e44e8-248f-4f50-a760-581972352af8 2021-12-06 20:05:39.732 7 INFO masakari.compute.nova [req-425752db-0599-42a8-bf2a-0229148410cc nova - - - -] Call get server command for instance 746178b2-14ce-4ce2-83f6-2bf9d613a887 2021-12-06 20:05:39.778 7 ERROR oslo.service.loopingcall [req-3d69363b-aa6c-4752-be52-285bc36f25c2 nova - - - -] Fixed interval looping call 'masakari.engine.drivers.taskflow.host_failure.EvacuateInstancesTask._evacuate_and_confirm.<locals>._wait_for_evacuation_confirmation' failed: masakari.exception.InstanceEvacuateFailed: Failed to evacuate instance 618e44e8-248f-4f50-a760-581972352af8 2021-12-06 20:05:39.778 7 ERROR oslo.service.loopingcall Traceback (most recent call last): 2021-12-06 20:05:39.778 7 ERROR oslo.service.loopingcall File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_service/loopingcall.py", line 150, in _run_loop 2021-12-06 20:05:39.778 7 ERROR oslo.service.loopingcall result = func(*self.args, **self.kw) 2021-12-06 20:05:39.778 7 ERROR oslo.service.loopingcall File "/var/lib/kolla/venv/lib/python3.8/site-packages/masakari/engine/drivers/taskflow/host_failure.py", line 207, in _wait_for_evacuation_confirmation 2021-12-06 20:05:39.778 7 ERROR oslo.service.loopingcall raise exception.InstanceEvacuateFailed( 2021-12-06 20:05:39.778 7 ERROR oslo.service.loopingcall masakari.exception.InstanceEvacuateFailed: Failed to evacuate instance 618e44e8-248f-4f50-a760-581972352af8 2021-12-06 20:05:39.778 7 ERROR oslo.service.loopingcall 2021-12-06 20:05:39.790 7 WARNING masakari.engine.drivers.taskflow.host_failure [req-d537ed5e-f5c8-4ba5-b559-0156883ca02b nova - - - -] Failed to evacuate instance 618e44e8-248f-4f50-a760-581972352af8: masakari.exception.InstanceEvacuateFailed: Failed to evacuate instance 618e44e8-248f-4f50-a760-581972352af8 2021-12-06 20:05:39.793 7 INFO masakari.compute.nova [req-c02a47f3-fc23-4d5b-9167-cf21bf8923db nova - - - -] Call unlock server command for instance 618e44e8-248f-4f50-a760-581972352af8 2021-12-06 20:05:40.422 7 ERROR oslo.service.loopingcall [req-425752db-0599-42a8-bf2a-0229148410cc nova - - - -] Fixed interval looping call 'masakari.engine.drivers.taskflow.host_failure.EvacuateInstancesTask._evacuate_and_confirm.<locals>._wait_for_evacuation_confirmation' failed: masakari.exception.InstanceEvacuateFailed: Failed to evacuate instance 746178b2-14ce-4ce2-83f6-2bf9d613a887 2021-12-06 20:05:40.422 7 ERROR oslo.service.loopingcall Traceback (most recent call last): 2021-12-06 20:05:40.422 7 ERROR oslo.service.loopingcall File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_service/loopingcall.py", line 150, in _run_loop 2021-12-06 20:05:40.422 7 ERROR oslo.service.loopingcall result = func(*self.args, **self.kw) 2021-12-06 20:05:40.422 7 ERROR oslo.service.loopingcall File "/var/lib/kolla/venv/lib/python3.8/site-packages/masakari/engine/drivers/taskflow/host_failure.py", line 207, in _wait_for_evacuation_confirmation 2021-12-06 20:05:40.422 7 ERROR oslo.service.loopingcall raise exception.InstanceEvacuateFailed( 2021-12-06 20:05:40.422 7 ERROR oslo.service.loopingcall masakari.exception.InstanceEvacuateFailed: Failed to evacuate instance 746178b2-14ce-4ce2-83f6-2bf9d613a887 2021-12-06 20:05:40.422 7 ERROR oslo.service.loopingcall 2021-12-06 20:05:40.423 7 WARNING masakari.engine.drivers.taskflow.host_failure [req-f8061ba7-353b-487c-8813-30f85216335f nova - - - -] Failed to evacuate instance 746178b2-14ce-4ce2-83f6-2bf9d613a887: masakari.exception.InstanceEvacuateFailed: Failed to evacuate instance 746178b2-14ce-4ce2-83f6-2bf9d613a887 2021-12-06 20:05:40.425 7 INFO masakari.compute.nova [req-7f0c16fe-8eaa-485b-a414-945fcd84a3c6 nova - - - -] Call unlock server command for instance 746178b2-14ce-4ce2-83f6-2bf9d613a887 2021-12-06 20:05:41.080 7 WARNING masakari.engine.drivers.taskflow.driver [req-49414eed-fdcf-4fd0-b803-e5cc5f2f9227 nova - - - -] Task 'EvacuateInstancesTask' (e49cc088-a9d7-4dd8-a6db-75339c6eaa4d) transitioned into state 'FAILURE' from state 'RUNNING' 4 predecessors (most recent first): Flow 'post_tasks' |__Flow 'main_tasks' |__Flow 'pre_tasks' |__Flow 'instance_evacuate_engine': masakari.exception.HostRecoveryFailureException: Failed to evacuate instances '618e44e8-248f-4f50-a760-581972352af8,746178b2-14ce-4ce2-83f6-2bf9d613a887' from host 'ctl02-hml.amt.net.br' 2021-12-06 20:05:41.080 7 ERROR masakari.engine.drivers.taskflow.driver Traceback (most recent call last): 2021-12-06 20:05:41.080 7 ERROR masakari.engine.drivers.taskflow.driver File "/var/lib/kolla/venv/lib/python3.8/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task 2021-12-06 20:05:41.080 7 ERROR masakari.engine.drivers.taskflow.driver result = task.execute(**arguments) 2021-12-06 20:05:41.080 7 ERROR masakari.engine.drivers.taskflow.driver File "/var/lib/kolla/venv/lib/python3.8/site-packages/masakari/engine/drivers/taskflow/host_failure.py", line 396, in execute 2021-12-06 20:05:41.080 7 ERROR masakari.engine.drivers.taskflow.driver _do_evacuate(self.context, host_name, instance_list) 2021-12-06 20:05:41.080 7 ERROR masakari.engine.drivers.taskflow.driver File "/var/lib/kolla/venv/lib/python3.8/site-packages/masakari/engine/drivers/taskflow/host_failure.py", line 376, in _do_evacuate 2021-12-06 20:05:41.080 7 ERROR masakari.engine.drivers.taskflow.driver raise exception.HostRecoveryFailureException( 2021-12-06 20:05:41.080 7 ERROR masakari.engine.drivers.taskflow.driver masakari.exception.HostRecoveryFailureException: Failed to evacuate instances '618e44e8-248f-4f50-a760-581972352af8,746178b2-14ce-4ce2-83f6-2bf9d613a887' from host 'ctl02-hml.amt.net.br' 2021-12-06 20:05:41.080 7 ERROR masakari.engine.drivers.taskflow.driver 2021-12-06 20:05:41.088 7 WARNING masakari.engine.drivers.taskflow.driver [req-49414eed-fdcf-4fd0-b803-e5cc5f2f9227 nova - - - -] Task 'EvacuateInstancesTask' (e49cc088-a9d7-4dd8-a6db-75339c6eaa4d) transitioned into state 'REVERTED' from state 'REVERTING' 2021-12-06 20:05:41.091 7 WARNING masakari.engine.drivers.taskflow.driver [req-49414eed-fdcf-4fd0-b803-e5cc5f2f9227 nova - - - -] Task 'PrepareHAEnabledInstancesTask' (21c32e80-7521-44bb-bd28-53f88c3d13da) transitioned into state 'REVERTED' from state 'REVERTING' 2021-12-06 20:05:41.093 7 WARNING masakari.engine.drivers.taskflow.driver [req-49414eed-fdcf-4fd0-b803-e5cc5f2f9227 nova - - - -] Task 'DisableComputeServiceTask' (b3522750-c7a6-4f71-ad8c-e2a61a40e2b8) transitioned into state 'REVERTED' from state 'REVERTING' 2021-12-06 20:05:41.095 7 WARNING masakari.engine.drivers.taskflow.driver [req-49414eed-fdcf-4fd0-b803-e5cc5f2f9227 nova - - - -] Flow 'instance_evacuate_engine' (3c8c5def-39b8-4956-8b2c-62a218e612ee) transitioned into state 'REVERTED' from state 'RUNNING' 2021-12-06 20:05:41.096 7 ERROR masakari.engine.manager [req-49414eed-fdcf-4fd0-b803-e5cc5f2f9227 nova - - - -] Failed to process notification '2468563d-70c5-41ef-ad04-a51b4ad3dd4d'. Reason: Failed to evacuate instances '618e44e8-248f-4f50-a760-581972352af8,746178b2-14ce-4ce2-83f6-2bf9d613a887' from host 'ctl02-hml.amt.net.br': masakari.exception.HostRecoveryFailureException: Failed to evacuate instances '618e44e8-248f-4f50-a760-581972352af8,746178b2-14ce-4ce2-83f6-2bf9d613a887' from host 'ctl02-hml.amt.net.br' 2021-12-06 20:05:41.099 7 INFO masakari.engine.manager [req-49414eed-fdcf-4fd0-b803-e5cc5f2f9227 nova - - - -] Notification 2468563d-70c5-41ef-ad04-a51b4ad3dd4d exits with status: error. 2021-12-06 20:07:08.803 7 INFO masakari.engine.manager [req-d2c18313-253f-4bb2-8abb-5ca5d8ac248a nova - - - -] Processing notification 2468563d-70c5-41ef-ad04-a51b4ad3dd4d of type: COMPUTE_HOST 2021-12-06 20:07:09.398 7 INFO masakari.compute.nova [req-5b595769-209f-422b-a71f-ef4507187a61 nova - - - -] Disable nova-compute on ctl02-hml.amt.net.br 2021-12-06 20:07:09.467 7 INFO masakari.engine.drivers.taskflow.host_failure [req-5b595769-209f-422b-a71f-ef4507187a61 nova - - - -] Sleeping 180 sec before starting recovery thread until nova recognizes the node down.2021-12-06 20:02:33.502 7 INFO masakari.engine.manager [req-034e0a9e-1b34-4db3-a2e0-6d61a369ba6c 6c128366b66346d38fb5493adf0cf666 e39a7ea7d17046b5b97b2253bf8195bc - - -] Processing notification 2468563d-70c5-41ef-ad04-a51b4ad3dd4d of type: COMPUTE_HOST 2021-12-06 20:02:34.052 7 INFO masakari.compute.nova [req-882d4151-0549-4cb2-acbb-90509073179c nova - - - -] Disable nova-compute on ctl02-hml.amt.net.br 2021-12-06 20:02:34.113 7 INFO masakari.engine.drivers.taskflow.host_failure [req-882d4151-0549-4cb2-acbb-90509073179c nova - - - -] Sleeping 180 sec before starting recovery thread until nova recognizes the node down. 2021-12-06 20:05:34.130 7 INFO masakari.compute.nova [req-c570bdef-0786-47bd-94fc-e6c1396eabb1 nova - - - -] Fetch Server list on ctl02-hml.amt.net.br 2021-12-06 20:05:35.166 7 INFO masakari.compute.nova [req-adec6e54-15ff-4a28-98c3-01070ba2cf87 nova - - - -] Call get server command for instance 618e44e8-248f-4f50-a760-581972352af8 2021-12-06 20:05:35.949 7 INFO masakari.compute.nova [req-49414eed-fdcf-4fd0-b803-e5cc5f2f9227 nova - - - -] Call get server command for instance 746178b2-14ce-4ce2-83f6-2bf9d613a887 2021-12-06 20:05:35.955 7 INFO masakari.compute.nova [req-e548525d-74f7-42c4-8045-2f63d645f76f nova - - - -] Call get server command for instance 618e44e8-248f-4f50-a760-581972352af8 2021-12-06 20:05:36.739 7 INFO masakari.compute.nova [req-9cea2ff2-0d32-42a1-8605-b889d95cadc0 nova - - - -] Call lock server command for instance 618e44e8-248f-4f50-a760-581972352af8 2021-12-06 20:05:36.747 7 INFO masakari.compute.nova [req-bc3bdea3-e923-4c08-bf84-bea711465dce nova - - - -] Call get server command for instance 746178b2-14ce-4ce2-83f6-2bf9d613a887 2021-12-06 20:05:37.391 7 INFO masakari.compute.nova [req-9cea2ff2-0d32-42a1-8605-b889d95cadc0 nova - - - -] Call evacuate command for instance 618e44e8-248f-4f50-a760-581972352af8 on host None 2021-12-06 20:05:37.478 7 INFO masakari.compute.nova [req-84d38844-ae87-4767-b822-0099bc40aa16 nova - - - -] Call lock server command for instance 746178b2-14ce-4ce2-83f6-2bf9d613a887 2021-12-06 20:05:38.059 7 INFO masakari.compute.nova [req-d57bd685-2e42-48e0-bf7d-9978ac516451 nova - - - -] Call get server command for instance 618e44e8-248f-4f50-a760-581972352af8 2021-12-06 20:05:38.102 7 INFO masakari.compute.nova [req-84d38844-ae87-4767-b822-0099bc40aa16 nova - - - -] Call evacuate command for instance 746178b2-14ce-4ce2-83f6-2bf9d613a887 on host None 2021-12-06 20:05:38.734 7 INFO masakari.compute.nova [req-25b4bb9c-2e05-409a-bae8-fd7c8348ba5b nova - - - -] Call get server command for instance 746178b2-14ce-4ce2-83f6-2bf9d613a887 2021-12-06 20:05:39.062 7 INFO masakari.compute.nova [req-3d69363b-aa6c-4752-be52-285bc36f25c2 nova - - - -] Call get server command for instance 618e44e8-248f-4f50-a760-581972352af8 2021-12-06 20:05:39.732 7 INFO masakari.compute.nova [req-425752db-0599-42a8-bf2a-0229148410cc nova - - - -] Call get server command for instance 746178b2-14ce-4ce2-83f6-2bf9d613a887 2021-12-06 20:05:39.778 7 ERROR oslo.service.loopingcall [req-3d69363b-aa6c-4752-be52-285bc36f25c2 nova - - - -] Fixed interval looping call 'masakari.engine.drivers.taskflow.host_failure.EvacuateInstancesTask._evacuate_and_confirm.<locals>._wait_for_evacuation_confirmation' failed: masakari.exception.InstanceEvacuateFailed: Failed to evacuate instance 618e44e8-248f-4f50-a760-581972352af8 2021-12-06 20:05:39.778 7 ERROR oslo.service.loopingcall Traceback (most recent call last): 2021-12-06 20:05:39.778 7 ERROR oslo.service.loopingcall File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_service/loopingcall.py", line 150, in _run_loop 2021-12-06 20:05:39.778 7 ERROR oslo.service.loopingcall result = func(*self.args, **self.kw) 2021-12-06 20:05:39.778 7 ERROR oslo.service.loopingcall File "/var/lib/kolla/venv/lib/python3.8/site-packages/masakari/engine/drivers/taskflow/host_failure.py", line 207, in _wait_for_evacuation_confirmation 2021-12-06 20:05:39.778 7 ERROR oslo.service.loopingcall raise exception.InstanceEvacuateFailed( 2021-12-06 20:05:39.778 7 ERROR oslo.service.loopingcall masakari.exception.InstanceEvacuateFailed: Failed to evacuate instance 618e44e8-248f-4f50-a760-581972352af8 2021-12-06 20:05:39.778 7 ERROR oslo.service.loopingcall 2021-12-06 20:05:39.790 7 WARNING masakari.engine.drivers.taskflow.host_failure [req-d537ed5e-f5c8-4ba5-b559-0156883ca02b nova - - - -] Failed to evacuate instance 618e44e8-248f-4f50-a760-581972352af8: masakari.exception.InstanceEvacuateFailed: Failed to evacuate instance 618e44e8-248f-4f50-a760-581972352af8 2021-12-06 20:05:39.793 7 INFO masakari.compute.nova [req-c02a47f3-fc23-4d5b-9167-cf21bf8923db nova - - - -] Call unlock server command for instance 618e44e8-248f-4f50-a760-581972352af8 2021-12-06 20:05:40.422 7 ERROR oslo.service.loopingcall [req-425752db-0599-42a8-bf2a-0229148410cc nova - - - -] Fixed interval looping call 'masakari.engine.drivers.taskflow.host_failure.EvacuateInstancesTask._evacuate_and_confirm.<locals>._wait_for_evacuation_confirmation' failed: masakari.exception.InstanceEvacuateFailed: Failed to evacuate instance 746178b2-14ce-4ce2-83f6-2bf9d613a887 2021-12-06 20:05:40.422 7 ERROR oslo.service.loopingcall Traceback (most recent call last): 2021-12-06 20:05:40.422 7 ERROR oslo.service.loopingcall File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_service/loopingcall.py", line 150, in _run_loop 2021-12-06 20:05:40.422 7 ERROR oslo.service.loopingcall result = func(*self.args, **self.kw) 2021-12-06 20:05:40.422 7 ERROR oslo.service.loopingcall File "/var/lib/kolla/venv/lib/python3.8/site-packages/masakari/engine/drivers/taskflow/host_failure.py", line 207, in _wait_for_evacuation_confirmation 2021-12-06 20:05:40.422 7 ERROR oslo.service.loopingcall raise exception.InstanceEvacuateFailed( 2021-12-06 20:05:40.422 7 ERROR oslo.service.loopingcall masakari.exception.InstanceEvacuateFailed: Failed to evacuate instance 746178b2-14ce-4ce2-83f6-2bf9d613a887 2021-12-06 20:05:40.422 7 ERROR oslo.service.loopingcall 2021-12-06 20:05:40.423 7 WARNING masakari.engine.drivers.taskflow.host_failure [req-f8061ba7-353b-487c-8813-30f85216335f nova - - - -] Failed to evacuate instance 746178b2-14ce-4ce2-83f6-2bf9d613a887: masakari.exception.InstanceEvacuateFailed: Failed to evacuate instance 746178b2-14ce-4ce2-83f6-2bf9d613a887 2021-12-06 20:05:40.425 7 INFO masakari.compute.nova [req-7f0c16fe-8eaa-485b-a414-945fcd84a3c6 nova - - - -] Call unlock server command for instance 746178b2-14ce-4ce2-83f6-2bf9d613a887 2021-12-06 20:05:41.080 7 WARNING masakari.engine.drivers.taskflow.driver [req-49414eed-fdcf-4fd0-b803-e5cc5f2f9227 nova - - - -] Task 'EvacuateInstancesTask' (e49cc088-a9d7-4dd8-a6db-75339c6eaa4d) transitioned into state 'FAILURE' from state 'RUNNING' 4 predecessors (most recent first): Flow 'post_tasks' |__Flow 'main_tasks' |__Flow 'pre_tasks' |__Flow 'instance_evacuate_engine': masakari.exception.HostRecoveryFailureException: Failed to evacuate instances '618e44e8-248f-4f50-a760-581972352af8,746178b2-14ce-4ce2-83f6-2bf9d613a887' from host 'ctl02-hml.amt.net.br' 2021-12-06 20:05:41.080 7 ERROR masakari.engine.drivers.taskflow.driver Traceback (most recent call last): 2021-12-06 20:05:41.080 7 ERROR masakari.engine.drivers.taskflow.driver File "/var/lib/kolla/venv/lib/python3.8/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task 2021-12-06 20:05:41.080 7 ERROR masakari.engine.drivers.taskflow.driver result = task.execute(**arguments) 2021-12-06 20:05:41.080 7 ERROR masakari.engine.drivers.taskflow.driver File "/var/lib/kolla/venv/lib/python3.8/site-packages/masakari/engine/drivers/taskflow/host_failure.py", line 396, in execute 2021-12-06 20:05:41.080 7 ERROR masakari.engine.drivers.taskflow.driver _do_evacuate(self.context, host_name, instance_list) 2021-12-06 20:05:41.080 7 ERROR masakari.engine.drivers.taskflow.driver File "/var/lib/kolla/venv/lib/python3.8/site-packages/masakari/engine/drivers/taskflow/host_failure.py", line 376, in _do_evacuate 2021-12-06 20:05:41.080 7 ERROR masakari.engine.drivers.taskflow.driver raise exception.HostRecoveryFailureException( 2021-12-06 20:05:41.080 7 ERROR masakari.engine.drivers.taskflow.driver masakari.exception.HostRecoveryFailureException: Failed to evacuate instances '618e44e8-248f-4f50-a760-581972352af8,746178b2-14ce-4ce2-83f6-2bf9d613a887' from host 'ctl02-hml.amt.net.br' 2021-12-06 20:05:41.080 7 ERROR masakari.engine.drivers.taskflow.driver 2021-12-06 20:05:41.088 7 WARNING masakari.engine.drivers.taskflow.driver [req-49414eed-fdcf-4fd0-b803-e5cc5f2f9227 nova - - - -] Task 'EvacuateInstancesTask' (e49cc088-a9d7-4dd8-a6db-75339c6eaa4d) transitioned into state 'REVERTED' from state 'REVERTING' 2021-12-06 20:05:41.091 7 WARNING masakari.engine.drivers.taskflow.driver [req-49414eed-fdcf-4fd0-b803-e5cc5f2f9227 nova - - - -] Task 'PrepareHAEnabledInstancesTask' (21c32e80-7521-44bb-bd28-53f88c3d13da) transitioned into state 'REVERTED' from state 'REVERTING' 2021-12-06 20:05:41.093 7 WARNING masakari.engine.drivers.taskflow.driver [req-49414eed-fdcf-4fd0-b803-e5cc5f2f9227 nova - - - -] Task 'DisableComputeServiceTask' (b3522750-c7a6-4f71-ad8c-e2a61a40e2b8) transitioned into state 'REVERTED' from state 'REVERTING' 2021-12-06 20:05:41.095 7 WARNING masakari.engine.drivers.taskflow.driver [req-49414eed-fdcf-4fd0-b803-e5cc5f2f9227 nova - - - -] Flow 'instance_evacuate_engine' (3c8c5def-39b8-4956-8b2c-62a218e612ee) transitioned into state 'REVERTED' from state 'RUNNING' 2021-12-06 20:05:41.096 7 ERROR masakari.engine.manager [req-49414eed-fdcf-4fd0-b803-e5cc5f2f9227 nova - - - -] Failed to process notification '2468563d-70c5-41ef-ad04-a51b4ad3dd4d'. Reason: Failed to evacuate instances '618e44e8-248f-4f50-a760-581972352af8,746178b2-14ce-4ce2-83f6-2bf9d613a887' from host 'ctl02-hml.amt.net.br': masakari.exception.HostRecoveryFailureException: Failed to evacuate instances '618e44e8-248f-4f50-a760-581972352af8,746178b2-14ce-4ce2-83f6-2bf9d613a887' from host 'ctl02-hml.amt.net.br' 2021-12-06 20:05:41.099 7 INFO masakari.engine.manager [req-49414eed-fdcf-4fd0-b803-e5cc5f2f9227 nova - - - -] Notification 2468563d-70c5-41ef-ad04-a51b4ad3dd4d exits with status: error. 2021-12-06 20:07:08.803 7 INFO masakari.engine.manager [req-d2c18313-253f-4bb2-8abb-5ca5d8ac248a nova - - - -] Processing notification 2468563d-70c5-41ef-ad04-a51b4ad3dd4d of type: COMPUTE_HOST 2021-12-06 20:07:09.398 7 INFO masakari.compute.nova [req-5b595769-209f-422b-a71f-ef4507187a61 nova - - - -] Disable nova-compute on ctl02-hml.amt.net.br 2021-12-06 20:07:09.467 7 INFO masakari.engine.drivers.taskflow.host_failure [req-5b595769-209f-422b-a71f-ef4507187a61 nova - - - -] Sleeping 180 sec before starting recovery thread until nova recognizes the node down. I tested forcing a kernel panic on target node Em sáb., 4 de dez. de 2021 às 08:48, Radosław Piliszek < radoslaw.piliszek@gmail.com> escreveu:
What do the Masakari API and Masakari Engine logs show?
-yoctozepto
On Fri, 3 Dec 2021 at 19:57, Rodrigo Lima <rodrigo.lima@o2sistemas.com> wrote:
I'm working on upgrading an openstack farm from victoria to wallaby. After successful upgrade, I would like to enable hacluster and masakari to test HA between failing compute nodes. Everything seems to be running (pacemaker with the remote nodes OK, corosync without errors, masakari-monitors detects the 2 compute nodes online), but... when I simulate the failure of a node with shutdown, the failure and notification appears in the hostmonitor log, but the instances that were on the failed node don't evacuate, and I couldn't find documentation that explains how to do this specific configuration, if necessary. Does anyone have any ideas?