[kolla] [nova] Duplicated compute service IDs in Openstack client
We're running kolla-ansible Train. We were trying to delete some compute services today and we got an error: $ openstack compute service delete 54 Failed to delete compute service with ID '54': Service id 54 refers to multiple services. (HTTP 400) (Request-ID: req-3bc400a4-367e-41dc-85f8-fa9011cc96cc) 1 of 1 compute services failed to delete. When we used "openstack compute service list" we saw that ID 54 was assigned to a nova-compute service on a hypervisor, and to a nova-scheduler service on a controller. We worked around it by using "nova service-list" to get the UUID-style ID and then "nova service-delete" After that I sorted by ID with " --sort-column ID" and found that some active services have duplicate IDs: | 33 | nova-compute |<region>-compute504.<domain> | AZ22 | enabled | up | 2022-06-27T18:22:47.000000 | | 33 | nova-scheduler | <region>-ctrl1.<domain> | internal | enabled | up | 2022-06-27T18:22:43.000000 | Is this a bug in the Openstack client?
Hello Albert, You are seeing conflicting IDs because these services belong to different Nova cells and are in different databases: your scheduler services are in cell0 while your compute hosts are probably in a default cell. This is why, starting from Nova API 2.53 [1], compute services are identified by UUID instead of database IDs. This is not really a bug with the OpenStack client, but a known issue that it uses the 2.1 API by default, with all its limitations. You can specify a newer API version like this: openstack --os-compute-api-version 2.53 compute service list openstack --os-compute-api-version 2.53 compute service delete <uuid> [1] https://docs.openstack.org/nova/latest/reference/api-microversion-history.ht... On Mon, 27 Jun 2022 at 21:02, Albert Braden <ozzzo@yahoo.com> wrote:
We're running kolla-ansible Train. We were trying to delete some compute services today and we got an error:
$ openstack compute service delete 54 Failed to delete compute service with ID '54': Service id 54 refers to multiple services. (HTTP 400) (Request-ID: req-3bc400a4-367e-41dc-85f8-fa9011cc96cc) 1 of 1 compute services failed to delete.
When we used "openstack compute service list" we saw that ID 54 was assigned to a nova-compute service on a hypervisor, and to a nova-scheduler service on a controller. We worked around it by using "nova service-list" to get the UUID-style ID and then "nova service-delete"
After that I sorted by ID with " --sort-column ID" and found that some active services have duplicate IDs:
| 33 | nova-compute |<region>-compute504.<domain> | AZ22 | enabled | up | 2022-06-27T18:22:47.000000 | | 33 | nova-scheduler | <region>-ctrl1.<domain> | internal | enabled | up | 2022-06-27T18:22:43.000000 |
Is this a bug in the Openstack client?
On Tue, 2022-06-28 at 10:03 +0200, Pierre Riteau wrote:
Hello Albert,
You are seeing conflicting IDs because these services belong to different Nova cells and are in different databases: your scheduler services are in cell0 while your compute hosts are probably in a default cell. This is why, starting from Nova API 2.53 [1], compute services are identified by UUID instead of database IDs.
This is not really a bug with the OpenStack client, but a known issue that it uses the 2.1 API by default, with all its limitations. You can specify a newer API version like this:
openstack --os-compute-api-version 2.53 compute service list
openstack --os-compute-api-version 2.53 compute service delete <uuid>
[1] https://docs.openstack.org/nova/latest/reference/api-microversion-history.ht... yep exactly its not a bug in nova or the client its not even really a know issue its how it was intended to work.
the old nova clint help test used to call out the duplicates for multi cell envs usage: nova service-delete <id> Delete the service by integer ID. If deleting a nova-compute service, be sure to stop the actual nova- compute process on the physical host before deleting the service with this command. Failing to do so can lead to the running service re-creating orphaned compute_nodes table records in the database. (Supported by API versions '2.0' - '2.latest') Positional arguments: <id> ID of service as an integer. Note that this may not uniquely identify a service in a multi-cell deployment. the openstack client (which you shoudl use) does not call that out but its equally true. openstack --os-compute-api-version 2.1 help service delete usage: openstack service delete [-h] <service> [<service> ...] Delete service(s) positional arguments: <service> Service(s) to delete (type, name or ID) optional arguments: -h, --help show this help message and exit you should prefer the uuid and its requried in a cellv2 deployument in most casess.
On Mon, 27 Jun 2022 at 21:02, Albert Braden <ozzzo@yahoo.com> wrote:
We're running kolla-ansible Train. We were trying to delete some compute services today and we got an error:
$ openstack compute service delete 54 Failed to delete compute service with ID '54': Service id 54 refers to multiple services. (HTTP 400) (Request-ID: req-3bc400a4-367e-41dc-85f8-fa9011cc96cc) 1 of 1 compute services failed to delete.
When we used "openstack compute service list" we saw that ID 54 was assigned to a nova-compute service on a hypervisor, and to a nova-scheduler service on a controller. We worked around it by using "nova service-list" to get the UUID-style ID and then "nova service-delete"
After that I sorted by ID with " --sort-column ID" and found that some active services have duplicate IDs:
33 | nova-compute |<region>-compute504.<domain> | AZ22 | enabled | up | 2022-06-27T18:22:47.000000 | 33 | nova-scheduler | <region>-ctrl1.<domain> | internal | enabled | up 2022-06-27T18:22:43.000000 |
Is this a bug in the Openstack client?
participants (3)
-
Albert Braden
-
Pierre Riteau
-
Sean Mooney