Open Stack

I have seen deadlocks in libvirt that could cause this. When you are in this state, check to see if you can do a virsh list on the node. If not, libvirt is deadlocked, and ubuntu may need to pull in a fix/newer version.

Vish

On Dec 12, 2014, at 2:12 PM, pcrews <gleebix at gmail.com> wrote:

> On 12/09/2014 03:54 PM, Ken'ichi Ohmichi wrote:
>> Hi,
>> 
>> This case is always tested by Tempest on the gate.
>> 
>> https://github.com/openstack/tempest/blob/master/tempest/api/compute/servers/test_delete_server.py#L152
>> 
>> So I guess this problem wouldn't happen on the latest version at least.
>> 
>> Thanks
>> Ken'ichi Ohmichi
>> 
>> ---
>> 
>> 2014-12-10 6:32 GMT+09:00 Joe Gordon <joe.gordon0 at gmail.com>:
>>> 
>>> 
>>> On Sat, Dec 6, 2014 at 5:08 PM, Danny Choi (dannchoi) <dannchoi at cisco.com>
>>> wrote:
>>>> 
>>>> Hi,
>>>> 
>>>> I have a VM which is in ERROR state.
>>>> 
>>>> 
>>>> +--------------------------------------+----------------------------------------------+--------+------------+-------------+--------------------+
>>>> 
>>>> | ID                                   | Name
>>>> | Status | Task State | Power State | Networks           |
>>>> 
>>>> 
>>>> +--------------------------------------+----------------------------------------------+--------+------------+-------------+--------------------+
>>>> 
>>>> | 1cb5bf96-619c-4174-baae-dd0d8c3d40c5 |
>>>> cirros--1cb5bf96-619c-4174-baae-dd0d8c3d40c5 | ERROR  | -          | NOSTATE
>>>> |                    |
>>>> 
>>>> 
>>>> I tried in both CLI “nova delete” and Horizon “terminate instance”.
>>>> Both accepted the delete command without any error.
>>>> However, the VM never got deleted.
>>>> 
>>>> Is there a way to remove the VM?
>>> 
>>> 
>>> What version of nova are you using? This is definitely a serious bug, you
>>> should be able to delete an instance in error state. Can you file a bug that
>>> includes steps on how to reproduce the bug along with all relevant logs.
>>> 
>>> bugs.launchpad.net/nova
>>> 
>>>> 
>>>> 
>>>> Thanks,
>>>> Danny
>>>> 
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>> 
>>> 
>>> 
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> 
>> 
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
> Hi,
> 
> I've encountered this in my own testing and have found that it appears to be tied to libvirt.
> 
> When I hit this, reset-state as the admin user reports success (and state is set), *but* things aren't really working as advertised and subsequent attempts to do anything with the errant vm's will send them right back into 'FLAIL' / can't delete / endless DELETING mode.
> 
> restarting libvirt-bin on my machine fixes this - after restart, the deleting vm's are properly wiped without any further user input to nova/horizon and all seems right in the world.
> 
> using:
> devstack
> ubuntu 14.04
> libvirtd (libvirt) 1.2.2
> 
> triggered via:
> lots of random create/reboot/resize/delete requests of varying validity and sanity.
> 
> Am in the process of cleaning up my test code so as not to hurt anyone's brain with the ugly and will file a bug once done, but thought this worth sharing.
> 
> Thanks,
> Patrick
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev