Problem with ressource provider
Hello, In our cluster, we have 4 computes running and we have an issue with the number 4. We can't create VMs on it, we can't migrate VMs to or from that node. VMs are still perfectly working though. After a first diagnosis, it appears that there's a problem with the ressource provider. Node is declared in the db with: - name: os-compute-4, uuid: d12ea77b-d678-40ce-a813-d8094cabbbd8 Here are the ressource provider: - name: os-compute-4, uuid: a9dc2a56-5b2d-49b1-ac47-6d996d2d029a - name: os-compute-4.openstack.local, uuid: d12ea776-d678-40ce-a813-d8094cabbbd8 In our opinion, os-compute-4.openstack.local shouldn't be there at all. We want to destroy both of the ressource provider and recreate one. I must also precise that os-compute-4 ressource provider has 0 allocation and os-compute-4.openstack.local only 3 (there’s at least 50 VMs running on it…). Moreover, for these 3 allocations, the server uuid doesn't correspond to any existing VMs. Overall, none of the VMs has a ressource allocation on os-compute-4. We found the command nova-manage placement heal_allocations on the Internet but we can't find it in any container, maybe deprecated ? The cluster is running Ussuri installed with Openstack-ansible. If you have any suggestion, any help would be appreciated. Thanks. :) Best, Marc-Antoine Godde
It would be easier to check resource provider by openstack cli, than looking into db. What's the name, short or FDQN, used by other compute nodes? Restart nova-compute and look into log, see which name is used to register resource provider. Tony ________________________________________ From: Marc-Antoine Godde (Student at CentraleSupelec) <marc-antoine.godde@student-cs.fr> Sent: January 18, 2022 01:56 PM To: openstack-discuss@lists.openstack.org Subject: Problem with ressource provider Hello, In our cluster, we have 4 computes running and we have an issue with the number 4. We can't create VMs on it, we can't migrate VMs to or from that node. VMs are still perfectly working though. After a first diagnosis, it appears that there's a problem with the ressource provider. Node is declared in the db with: - name: os-compute-4, uuid: d12ea77b-d678-40ce-a813-d8094cabbbd8 Here are the ressource provider: - name: os-compute-4, uuid: a9dc2a56-5b2d-49b1-ac47-6d996d2d029a - name: os-compute-4.openstack.local, uuid: d12ea776-d678-40ce-a813-d8094cabbbd8 In our opinion, os-compute-4.openstack.local shouldn't be there at all. We want to destroy both of the ressource provider and recreate one. I must also precise that os-compute-4 ressource provider has 0 allocation and os-compute-4.openstack.local only 3 (there’s at least 50 VMs running on it…). Moreover, for these 3 allocations, the server uuid doesn't correspond to any existing VMs. Overall, none of the VMs has a ressource allocation on os-compute-4. We found the command nova-manage placement heal_allocations on the Internet but we can't find it in any container, maybe deprecated ? The cluster is running Ussuri installed with Openstack-ansible. If you have any suggestion, any help would be appreciated. Thanks. :) Best, Marc-Antoine Godde
Here is what we get. :) Thanks for your help [cid:9921FF69-0B8A-456C-969F-2261C88048E0] Le 18 janv. 2022 à 23:17, Tony Liu <tonyliu0592@hotmail.com<mailto:tonyliu0592@hotmail.com>> a écrit : It would be easier to check resource provider by openstack cli, than looking into db. What's the name, short or FDQN, used by other compute nodes? Restart nova-compute and look into log, see which name is used to register resource provider. Tony ________________________________________ From: Marc-Antoine Godde (Student at CentraleSupelec) <marc-antoine.godde@student-cs.fr<mailto:marc-antoine.godde@student-cs.fr>> Sent: January 18, 2022 01:56 PM To: openstack-discuss@lists.openstack.org<mailto:openstack-discuss@lists.openstack.org> Subject: Problem with ressource provider Hello, In our cluster, we have 4 computes running and we have an issue with the number 4. We can't create VMs on it, we can't migrate VMs to or from that node. VMs are still perfectly working though. After a first diagnosis, it appears that there's a problem with the ressource provider. Node is declared in the db with: - name: os-compute-4, uuid: d12ea77b-d678-40ce-a813-d8094cabbbd8 Here are the ressource provider: - name: os-compute-4, uuid: a9dc2a56-5b2d-49b1-ac47-6d996d2d029a - name: os-compute-4.openstack.local, uuid: d12ea776-d678-40ce-a813-d8094cabbbd8 In our opinion, os-compute-4.openstack.local shouldn't be there at all. We want to destroy both of the ressource provider and recreate one. I must also precise that os-compute-4 ressource provider has 0 allocation and os-compute-4.openstack.local only 3 (there’s at least 50 VMs running on it…). Moreover, for these 3 allocations, the server uuid doesn't correspond to any existing VMs. Overall, none of the VMs has a ressource allocation on os-compute-4. We found the command nova-manage placement heal_allocations on the Internet but we can't find it in any container, maybe deprecated ? The cluster is running Ussuri installed with Openstack-ansible. If you have any suggestion, any help would be appreciated. Thanks. :) Best, Marc-Antoine Godde
Check /etc/hosts and /etc/hostname in all 4 compute nodes and ensure they are consistent. Check nova-compute logging to see which name is used as the provider. If you are sure the provide is safe to be deleted, you can remove it by openstack cli. Tony ________________________________________ From: Marc-Antoine Godde (Student at CentraleSupelec) <marc-antoine.godde@student-cs.fr> Sent: January 18, 2022 02:25 PM To: Tony Liu Cc: openstack-discuss@lists.openstack.org Subject: Re: Problem with ressource provider Here is what we get. :) Thanks for your help [cid:9921FF69-0B8A-456C-969F-2261C88048E0] Le 18 janv. 2022 à 23:17, Tony Liu <tonyliu0592@hotmail.com<mailto:tonyliu0592@hotmail.com>> a écrit : It would be easier to check resource provider by openstack cli, than looking into db. What's the name, short or FDQN, used by other compute nodes? Restart nova-compute and look into log, see which name is used to register resource provider. Tony ________________________________________ From: Marc-Antoine Godde (Student at CentraleSupelec) <marc-antoine.godde@student-cs.fr<mailto:marc-antoine.godde@student-cs.fr>> Sent: January 18, 2022 01:56 PM To: openstack-discuss@lists.openstack.org<mailto:openstack-discuss@lists.openstack.org> Subject: Problem with ressource provider Hello, In our cluster, we have 4 computes running and we have an issue with the number 4. We can't create VMs on it, we can't migrate VMs to or from that node. VMs are still perfectly working though. After a first diagnosis, it appears that there's a problem with the ressource provider. Node is declared in the db with: - name: os-compute-4, uuid: d12ea77b-d678-40ce-a813-d8094cabbbd8 Here are the ressource provider: - name: os-compute-4, uuid: a9dc2a56-5b2d-49b1-ac47-6d996d2d029a - name: os-compute-4.openstack.local, uuid: d12ea776-d678-40ce-a813-d8094cabbbd8 In our opinion, os-compute-4.openstack.local shouldn't be there at all. We want to destroy both of the ressource provider and recreate one. I must also precise that os-compute-4 ressource provider has 0 allocation and os-compute-4.openstack.local only 3 (there’s at least 50 VMs running on it…). Moreover, for these 3 allocations, the server uuid doesn't correspond to any existing VMs. Overall, none of the VMs has a ressource allocation on os-compute-4. We found the command nova-manage placement heal_allocations on the Internet but we can't find it in any container, maybe deprecated ? The cluster is running Ussuri installed with Openstack-ansible. If you have any suggestion, any help would be appreciated. Thanks. :) Best, Marc-Antoine Godde
I will do so. Should I let nova recreate the provider by restarting nova and then manually add allocation to VMs on the new provider ? Marc-Antoine
Le 18 janv. 2022 à 23:30, Tony Liu <tonyliu0592@hotmail.com> a écrit :
Check /etc/hosts and /etc/hostname in all 4 compute nodes and ensure they are consistent.
Check nova-compute logging to see which name is used as the provider.
If you are sure the provide is safe to be deleted, you can remove it by openstack cli.
Tony
________________________________________ From: Marc-Antoine Godde (Student at CentraleSupelec) <marc-antoine.godde@student-cs.fr> Sent: January 18, 2022 02:25 PM To: Tony Liu Cc: openstack-discuss@lists.openstack.org Subject: Re: Problem with ressource provider
Here is what we get. :)
Thanks for your help
[cid:9921FF69-0B8A-456C-969F-2261C88048E0]
Le 18 janv. 2022 à 23:17, Tony Liu <tonyliu0592@hotmail.com<mailto:tonyliu0592@hotmail.com>> a écrit :
It would be easier to check resource provider by openstack cli, than looking into db.
What's the name, short or FDQN, used by other compute nodes? Restart nova-compute and look into log, see which name is used to register resource provider.
Tony ________________________________________ From: Marc-Antoine Godde (Student at CentraleSupelec) <marc-antoine.godde@student-cs.fr<mailto:marc-antoine.godde@student-cs.fr>> Sent: January 18, 2022 01:56 PM To: openstack-discuss@lists.openstack.org<mailto:openstack-discuss@lists.openstack.org> Subject: Problem with ressource provider
Hello,
In our cluster, we have 4 computes running and we have an issue with the number 4.
We can't create VMs on it, we can't migrate VMs to or from that node. VMs are still perfectly working though. After a first diagnosis, it appears that there's a problem with the ressource provider.
Node is declared in the db with: - name: os-compute-4, uuid: d12ea77b-d678-40ce-a813-d8094cabbbd8
Here are the ressource provider:
- name: os-compute-4, uuid: a9dc2a56-5b2d-49b1-ac47-6d996d2d029a
- name: os-compute-4.openstack.local, uuid: d12ea776-d678-40ce-a813-d8094cabbbd8
In our opinion, os-compute-4.openstack.local shouldn't be there at all. We want to destroy both of the ressource provider and recreate one.
I must also precise that os-compute-4 ressource provider has 0 allocation and os-compute-4.openstack.local only 3 (there’s at least 50 VMs running on it…). Moreover, for these 3 allocations, the server uuid doesn't correspond to any existing VMs. Overall, none of the VMs has a ressource allocation on os-compute-4.
We found the command nova-manage placement heal_allocations on the Internet but we can't find it in any container, maybe deprecated ? The cluster is running Ussuri installed with Openstack-ansible.
If you have any suggestion, any help would be appreciated. Thanks. :)
Best, Marc-Antoine Godde
<virt.png>
participants (2)
-
Marc-Antoine Godde (Student at CentraleSupelec)
-
Tony Liu