VM stuck in hard_reboot state

Fabian Zimmermann dev.faz at gmail.com
Fri Oct 23 12:29:57 UTC 2020


Hi,

just forwarding your answer to the list - sorry for not using reply-all :(

Is your nova-compute-agent on the hypervisor working as expected? (check
the logs, restart)

Are you able to reset the state?
You should ensure the instance is not running somewhere else!

https://docs.openstack.org/ocata/admin-guide/support-compute.html#reset-the-state-of-an-instance

 Fabian


Am 23.10.20 um 14:16 schrieb Gregory SACRE:
> Here they are:
> 
> compute $ virsh list
>  Id   Name   State
> --------------------
> 
> compute $ virsh list --all
>  Id   Name                State
> ------------------------------------
>  -    instance-00000008   shut off
>  -    instance-00000009   shut off
>  -    instance-00000015   shut off
>  -    instance-00000016   shut off
>  -    instance-00000017   shut off
>  -    instance-0000001c   shut off
>  -    instance-0000001d   shut off
>  -    instance-0000001e   shut off
>  -    instance-0000001f   shut off
> 
> controller $ openstack server show mediawiki
> +-----------------------------+----------------------------------------------------------+
> | Field                       | Value
>                   |
> +-----------------------------+----------------------------------------------------------+
> | OS-DCF:diskConfig           | AUTO
>                   |
> | OS-EXT-AZ:availability_zone | nova
>                   |
> | OS-EXT-STS:power_state      | Running
>                   |
> | OS-EXT-STS:task_state       | rebooting_hard
>                   |
> | OS-EXT-STS:vm_state         | active
>                   |
> | OS-SRV-USG:launched_at      | 2019-08-09T13:43:06.000000
>                   |
> | OS-SRV-USG:terminated_at    | None
>                   |
> | accessIPv4                  |
>                   |
> | accessIPv6                  |
>                   |
> | addresses                   | wiki-selfservice=172.16.1.54,
> 192.168.150.154            |
> | config_drive                |
>                   |
> | created                     | 2019-08-09T13:32:49Z
>                   |
> | flavor                      | m2.large (7)
>                   |
> | hostId                      |
> b8cbd9da0d28d67fff65b88405428269c97d8d27289e2f8ee35f5485 |
> | id                          | 0e3eafb6-1b36-4e03-8548-8a3294d82c65
>                   |
> | image                       |
>                   |
> | key_name                    | gsa-key
>                   |
> | name                        | mediawiki
>                   |
> | project_id                  | a5f2b77a4a8e499791ee981b991f2df3
>                   |
> | properties                  |
>                   |
> | security_groups             | name='teleport'
>                   |
> |                             | name='default'
>                   |
> |                             | name='global_http'
>                   |
> | status                      | HARD_REBOOT
>                   |
> | updated                     | 2020-10-23T08:50:09Z
>                   |
> | user_id                     | 16ea223cee1345dda504c8408c55fb2a
>                   |
> | volumes_attached            |
> id='bdfe4fc6-1e85-4b5d-b9b6-16147fbfc005'                |
> +-----------------------------+----------------------------------------------------------+
> 
> Thanks for the help!
> 
> 
> Sorry for this question, but shouldn't I reply to the mailing list?
> I'm new to this mailing list so I'm not 100% sure on the habits (I
> have read the mailing list netiquette, but I prefer to ask first :-))
> 
> 
> On Fri, Oct 23, 2020 at 12:26 PM Fabian Zimmermann <dev.faz at gmail.com> wrote:
>>
>> Hi,
>>
>> could you dump the outputs of
>>
>> virsh list
>> virsh list --all
>> openstack server show
>>
>>  Fabian
>>
>> Gregory SACRE <gregory.sacre at gmail.com> schrieb am Fr., 23. Okt. 2020, 12:16:
>>>
>>> Dear all,
>>>
>>>
>>> I've been using Openstack (Ubuntu 18.04 install following Openstack documentation) for about a year in a lab at work.
>>> I have one VM running 24/7 with Mediawiki on it (rest is for testing).
>>>
>>> It has been working fine until I updated packages yesterday. Here is the list of relevant packages that were updated:
>>>
>>> 2020-10-22 09:44:04 status installed neutron-common:all 2:14.3.1-0ubuntu1~cloud0
>>> 2020-10-22 09:44:06 status installed nova-common:all 2:19.3.1-0ubuntu1~cloud0
>>> 2020-10-22 09:44:12 status installed python3-nova:all 2:19.3.1-0ubuntu1~cloud0
>>> 2020-10-22 09:44:16 status installed python3-neutron:all 2:14.3.1-0ubuntu1~cloud0
>>> 2020-10-22 09:44:25 status installed neutron-linuxbridge-agent:all 2:14.3.1-0ubuntu1~cloud0
>>> 2020-10-22 09:44:29 status installed nova-compute-libvirt:all 2:19.3.1-0ubuntu1~cloud0
>>> 2020-10-22 09:44:35 status installed nova-compute-kvm:all 2:19.3.1-0ubuntu1~cloud0
>>> 2020-10-22 09:45:04 status installed nova-compute:all 2:19.3.1-0ubuntu1~cloud0
>>>
>>> Since then, the VM does not respond (ping, ssh...).
>>> I could list it (via Web interface or controller command line) and it showed as ACTIVE.
>>> I rebooted the server thinking maybe something was stuck. The server came back online, the VM still ACTIVE but cannot connect to it.
>>> My next step was to hard reboot the VM. Since then, the VM is in HARD_REBOOT state and would not move from there. I tried to shut it down, restart again the server, but no, nothing changes. When I try to shut it down it says it cannot as it is in hard reboot state.
>>>
>>> I would at least like to be able to save some stuff that's on it (I know, I should have backed it up before the update...).
>>> My Google searches on "openstack vm stuck hard reboot" did not help...
>>>
>>> Do not hesitate if you need more info.
>>> Any help would really be appreciated!
>>>
>>>
>>> Kr,
>>>
>>> Gregory



More information about the openstack-discuss mailing list