[nova] Clean up "building" instances
Eugen Block
eblock at nde.ag
Mon Feb 20 10:30:29 UTC 2023
Hi,
we had a network issue two weeks ago in a HA Victoria cloud resulting
in a couple of stale resources (in pending state). Most of them I
could cleanup relatively easy, but two instances are left in
"building" state, but not yet in the nova database so I can't just
remove them via 'openstack server delete <UUID>'. I've been looking
through the various nova databases where traces have been left to get
an impression where I could intervene (although I don't like
manipulating the database).
The VMs are two amphora instances:
control01:~ # openstack server list --project service | grep -v ACTIVE
+--------------------------------------+----------------------------------------------+--------+-------------------------------------------------------------------------------------+---------------------------+---------+
| ID | Name
| Status | Networks
| Image |
Flavor |
+--------------------------------------+----------------------------------------------+--------+-------------------------------------------------------------------------------------+---------------------------+---------+
| 0453a7e5-e4f9-419b-ad71-d837a20ef6bb |
amphora-0ee32901-0c59-4752-8253-35b66da176ea | BUILD |
| amphora-x64-haproxy_1.0.0 | amphora |
| dc8cdc3a-f6b2-469b-af6f-ba2aa130ea9b |
amphora-4990a47b-fe8a-431a-90ec-5ac2368a5251 | BUILD |
| amphora-x64-haproxy_1.0.0 | amphora
|+--------------------------------------+----------------------------------------------+--------+-------------------------------------------------------------------------------------+---------------------------+---------+
The database tables referring to the UUID
0453a7e5-e4f9-419b-ad71-d837a20ef6bb are these:
nova_cell0/instance_id_mappings.ibd
nova_cell0/instance_info_caches.ibd
nova_cell0/instance_extra.ibd
nova_cell0/instances.ibd
nova_cell0/instance_system_metadata.ibd
octavia/amphora.ibd
nova_api/instance_mappings.ibd
nova_api/request_specs.ibd
My first approach would be to update the nova_cell0.instances table
and edit the fields 'vm_state' and 'task_state', or even remove the
intire row. But I don't know about the implications this would have on
the other tables, so I'd like to know how you would recommend to deal
with these orphans. Any comment is appreciated!
Thanks,
Eugen
More information about the openstack-discuss
mailing list