Open Stack

Hi again,

I just wanted to get some clarification on how to proceed.

> what you proably need to do in this case is check if the RPs still  
> have allocations and if so
> verify that the allocation are owned by vms that nolonger exist.
> if that is the case you should be able to delete teh allcaotion and  
> then the RP
> if the allocations are related to active vms that are now on the  
> rebuild nodes then you will have to try and
> heal the allcoations.

I checked all allocations for the old compute nodes, those are all  
existing VMs. So simply deleting the allocations won't do any good, I  
guess. From [1] I understand that I should overwrite all allocations  
(we're on Train so there's no "unset" available yet) for those VMs to  
point to the new compute nodes (resource_providers). After that I  
should delete the resource providers, correct?
I ran "heal_allocations" for one uncritical instance, but it didn't  
have any visible effect, the allocations still show one of the old  
compute nodes.
What I haven't tried yet is to delete allocations for an instance and  
then try to heal it as the docs also mention.

Do I understand that correctly or am I still missing something?

Regards,
Eugen


[1]  
https://docs.openstack.org/nova/latest/admin/troubleshooting/orphaned-allocations.html

Zitat von Sean Mooney <smooney at redhat.com>:

> On Mon, 2021-03-08 at 14:18 +0000, Eugen Block wrote:
>> Thank you, Sean.
>>
>> > so you need to do
>> > openstack compute service list to get teh compute service ids
>> > then do
>> > openstack compute service delete <id-1> <id-2> ...
>> >
>> > you need to make sure that you only remvoe the unused old serivces
>> > but i think that would fix your issue.
>>
>> That's the thing, they don't show up in the compute service list. But
>> I also found them in the resource_providers table, only the old
>> compute nodes appear here:
>>
>> MariaDB [nova]> select name from nova_api.resource_providers;
>> +--------------------------+
>> > name                     |
>> +--------------------------+
>> > compute1.fqdn            |
>> > compute2.fqdn            |
>> > compute3.fqdn            |
>> > compute4.fqdn            |
>> +--------------------------+
> ah in that case the compute service delete is ment to remove the RPs too
> but if the RP had stale allcoation at teh time of the delete the RP  
> delete will fail
>
> what you proably need to do in this case is check if the RPs still  
> have allocations and if so
> verify that the allocation are owned by vms that nolonger exist.
> if that is the case you should be able to delete teh allcaotion and  
> then the RP
> if the allocations are related to active vms that are now on the  
> rebuild nodes then you will have to try and
> heal the allcoations.
>
> there is a openstack client extention called osc-placement that you  
> can install to help.
> we also have a heal allcoation command in nova-manage that may help  
> but the next step would be to validate
> if the old RPs are still in use or not. from there you can then work  
> to align novas and placment view with
> the real toplogy.
>
> that could invovle removing the old compute nodes form the  
> compute_nodes table or marking them as deleted but
> both nova db and plamcent need to be kept in sysnc to correct your  
> current issue.
>
>>
>>
>> Zitat von Sean Mooney <smooney at redhat.com>:
>>
>> > On Mon, 2021-03-08 at 13:18 +0000, Eugen Block wrote:
>> > > Hi *,
>> > >
>> > > I have a quick question, last year we migrated our OpenStack to a
>> > > highly available environment through a reinstall of all nodes. The
>> > > migration went quite well, we're working happily in the new cloud but
>> > > the databases still contain deprecated data. For example, the
>> > > nova-scheduler logs lines like these on a regular basis:
>> > >
>> > > /var/log/nova/nova-scheduler.log:2021-02-19 12:02:46.439 23540 WARNING
>> > > nova.scheduler.host_manager [...] No compute service record found for
>> > > host compute1
>> > >
>> > > This is one of the old compute nodes that has been reinstalled and is
>> > > now compute01. I tried to find the right spot to delete some lines in
>> > > the DB but there are a couple of places so I wanted to check and ask
>> > > you for some insights.
>> > >
>> > > The scheduler messages seem to originate in
>> > >
>> > > /usr/lib/python3.6/site-packages/nova/scheduler/host_manager.py
>> > >
>> > > ---snip---
>> > >          for cell_uuid, computes in compute_nodes.items():
>> > >              for compute in computes:
>> > >                  service = services.get(compute.host)
>> > >
>> > >                  if not service:
>> > >                      LOG.warning(
>> > >                          "No compute service record found for host
>> > > %(host)s",
>> > >                          {'host': compute.host})
>> > >                      continue
>> > > ---snip---
>> > >
>> > > So I figured it could be this table in the nova DB:
>> > >
>> > > ---snip---
>> > > MariaDB [nova]> select host,deleted from compute_nodes;
>> > > +-----------+---------+
>> > > > host      | deleted |
>> > > +-----------+---------+
>> > > > compute01 |       0 |
>> > > > compute02 |       0 |
>> > > > compute03 |       0 |
>> > > > compute04 |       0 |
>> > > > compute05 |       0 |
>> > > > compute1  |       0 |
>> > > > compute2  |       0 |
>> > > > compute3  |       0 |
>> > > > compute4  |       0 |
>> > > +-----------+---------+
>> > > ---snip---
>> > >
>> > > What would be the best approach here to clean up a little? I believe
>> > > it would be safe to simply purge those lines containing the old
>> > > compute node, but there might be a smoother way. Or maybe there are
>> > > more places to purge old data from?
>> > so the step you porably missed was deleting the old compute  
>> service records
>> >
>> > so you need to do
>> > openstack compute service list to get teh compute service ids
>> > then do
>> > openstack compute service delete <id-1> <id-2> ...
>> >
>> > you need to make sure that you only remvoe the unused old serivces
>> > but i think that would fix your issue.
>> >
>> > >
>> > > I'd appreciate any ideas.
>> > >
>> > > Regards,
>> > > Eugen
>> > >
>> > >
>>
>>
>>