Open Stack

Hi Masakari community,

I would like to get your help  to understand what may be causing the Masakari notification failed.  I do get  success cases which the engine got the notification, VM got shutdown and rebooted ok.

Having said that, there are some cases that the notification failed and it seems there are some conflicts going on. 20% to 40% chance.

Feb 20 21:53:21 masakari-2 masakari-engine[3807]: 2018-02-20 21:53:21.517 WARNING masakari.engine.drivers.taskflow.driver [req-ce909151-1afb-4f2f-abf4-f25d54f25c6b service None] Task 'masakari.engine.drivers.taskflow.instance_failure.StopInstanceTask;instance:recovery' (e85dec06-1498-482c-a63a-51f855745c32) transitioned into state 'FAILURE' from state 'RUNNING'
Feb 20 21:53:21 masakari-2 masakari-engine[3807]: 1 predecessors (most recent first):
Feb 20 21:53:21 masakari-2 masakari-engine[3807]:   Flow 'instance_recovery_engine': Conflict: Conflict

Is it normal that masakari notification would be failed because of timing or conflicting events? FYI, I only have one VM and one active notification.

Enclosed is the log file I got from the engine.

I do appreciate if anyone of you can provide some insight what to do with the failure. Any tip where to look at etc? Timeout?

Thanks.
Louie


| notification_uuid                    | generated_time             | status   | type | source_host_uuid                     | payload                                                                                                                                  |
+--------------------------------------+----------------------------+----------+------+--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+
| 42ccee84-0ea5-4163-84a5-028a0bb914a3 | 2018-02-20T21:52:03.000000 | failed   | VM   | 66c8b5b9-03f5-4843-8a9c-fa83af807a9b | {u'instance_uuid': u'565da9ba-3c0c-4087-83ca-32a5a1b00a55', u'vir_domain_event': u'STOPPED_FAILED', u'event': u'QEMU_GUEST_AGENT_ERROR'} |
| aa4184f3-b002-4ba8-a403-f22ccd4ce6b5 | 2018-02-20T21:42:54.000000 | finished | VM   | 66c8b5b9-03f5-4843-8a9c-fa83af807a9b | {u'instance_uuid': u'565da9ba-3c0c-4087-83ca-32a5a1b00a55', u'vir_domain_event': u'STOPPED_FAILED', u'event': u'QEMU_GUEST_AGENT_ERROR'} |

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180220/7f12b565/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: err.log
Type: application/octet-stream
Size: 269668 bytes
Desc: err.log
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180220/7f12b565/attachment-0001.obj>