[openstack-dev] Instance state recovery - LP bug 957009

Gurjar, Unmesh Unmesh.Gurjar at nttdata.com
Fri Nov 2 14:31:05 UTC 2012


Hi,

I would like to collect some thoughts for fixing LP bug: https://bugs.launchpad.net/nova/+bug/957009.

Following is the scenario:

1.       The Compute service hosting an active instance goes down and user requests DELETE operation on that instance.

2.       The Nova API finds the Compute service running (since it had updated heartbeat recently than 'service_down_time' period). It marks the instance task_state as DELETING and casts a delete to the Compute.

3.       If the RabbitMQ service happens to restart before the Compute service comes up, the DELETE message to Compute will be lost and the instance will remain in DELETING state (vm_state=ACTIVE, task_state=DELETING, power_state=1). Also, even the '_cleanup_running_deleted_instances' periodic task won't clean up such instance.

Solution:
I am planning to add a periodic task to Nova Compute similar to the '_check_instance_build_time' periodic task, which will poll for instances in 'DELETING' state for some configurable value and set them to ERROR state.

Suggestions are welcome.

Thanks & Regards,
Unmesh Gurjar | Lead Engineer | NTT DATA Global Technology Services Private Limited | w. +91.20.6604.1500 x 379 | m. +91.982.324.7631 | unmesh.gurjar at nttdata.com<mailto:unmesh.gurjar at nttdata.com> | Learn more at nttdata.com/americas


______________________________________________________________________
Disclaimer:This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data.  If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20121102/fdb20144/attachment.html>


More information about the OpenStack-dev mailing list