Open Stack

Yes, I specified the host, as I'm trying to test live-migration to a new host, running a new operating system (Ubuntu 20.04 vs CentOS 8.4).

I'm seeing the following in my logs during startup:
No provider configs found in /etc/nova/provider_config/
Could this have anything to do with this problem?

I haven't found a placement.log anywhere yet.  Where would I expect to find that?  Alongside nova-api, or alongside nova-compute?

Thank you,

Dominic L. Hilsbos, MBA
Vice President – Information Technology
Perform Air International Inc.
DHilsbos at PerformAir.com
www.PerformAir.com

-----Original Message-----
From: Sean Mooney [mailto:smooney at redhat.com] 
Sent: Thursday, July 29, 2021 5:54 AM
To: Eugen Block; openstack-discuss at lists.openstack.org
Subject: Re: [ops][nova][victoria] Unable to Migrate: Host filter ignoring hosts

On Thu, 2021-07-29 at 08:32 +0000, Eugen Block wrote:
> Do you see anything in the placement.log? Are there enough resources  
> on that node? If you have it installed you could check the current  
> allocations with the osc-placement command:
> 
> openstack resource provider list
> 
> and then
> 
> openstack resource provider show --allocations <UUID>

if we got to the filters the hosts passed placement resouce check initally but it could fail to claim if there was a concurrent boot or migration.
the host is being rejected by the host filter

looking at the it looks like you have 2 schduling reuests.
in the first request we claim the placment resouces and select host chd0p-oscp1

17 2021-07-28 13:43:32.488 661142 DEBUG nova.scheduler.utils [req-df136ad7-5c7c-4099-97c4-caf9853e9a48 d7c514813e5d4fe6815f5f59e8e35f2f a008ad02d16f436a9e320882ca497055 - default default] Attempting
to claim resources in the placement API for instance c44f86ec-8e21-472e-9200-94c21d52904f claim_resources /usr/lib/python3.6/site-packages/nova/scheduler/utils.py:1215
18 2021-07-28 13:43:32.972 661142 DEBUG nova.scheduler.filter_scheduler [req-df136ad7-5c7c-4099-97c4-caf9853e9a48 d7c514813e5d4fe6815f5f59e8e35f2f a008ad02d16f436a9e320882ca497055 - default default]
[instance: c44f86ec-8e21-472e-9200-94c21d52904f] Selected host: (chd0p-oscp1, chd0p-oscp1) ram: 257162MB disk: 6846464MB io_ops: 0 instances: 0 _consume_selected_host /usr/lib/python3.6/site-
packages/nova/scheduler/filter_scheduler.py:354

then there is a seond phase where the host filter eliminates the host

37 2021-07-28 13:43:34.641 661143 INFO nova.scheduler.host_manager [req-df136ad7-5c7c-4099-97c4-caf9853e9a48 d7c514813e5d4fe6815f5f59e8e35f2f a008ad02d16f436a9e320882ca497055 - default default] Host
filter only checking host chd0p-oscp1 and node chd0p-oscp1
38 2021-07-28 13:43:34.641 661143 INFO nova.scheduler.host_manager [req-df136ad7-5c7c-4099-97c4-caf9853e9a48 d7c514813e5d4fe6815f5f59e8e35f2f a008ad02d16f436a9e320882ca497055 - default default] Host
filter ignoring hosts: chd0p-oscp1

in this second phase chd0p-oscp1is on the ignored list and its the only one that is in the lsit. so there are no host left.

i suspect you specified chd0p-oscp1 as the destination of the live migration correct rather then allowing any host as the destination.

i would use the request id req-df136ad7-5c7c-4099-97c4-caf9853e9a48 to check the nova comppute log on chd0p-oscp1.
i suspect the migration was attempted and failed or the calim of the resouse on line 17 failed so it went to the next host.


we can see in the inital query the ingnored hosts was empty. it is only populated in this case if the host was eliminated because the claim faild or the migration failed.

06 2021-07-28 13:43:32.477 661142 INFO nova.scheduler.host_manager [req-df136ad7-5c7c-4099-97c4-caf9853e9a48 d7c514813e5d4fe6815f5f59e8e35f2f a008ad02d16f436a9e320882ca497055 - default default] Host
filter only checking host chd0p-oscp1 and node chd0p-oscp1
07 2021-07-28 13:43:32.478 661142 INFO nova.scheduler.host_manager [req-df136ad7-5c7c-4099-97c4-caf9853e9a48 d7c514813e5d4fe6815f5f59e8e35f2f a008ad02d16f436a9e320882ca497055 - default default] Host
filter ignoring hosts: 

so i suspect Eugen is right that its the placment claim that is failing or the migration failed.

if the request id show up in the nova comppute log on chd0p-oscp1 then the allocation claim in placment suceeded if not then we know it was the placment claim that failed.

you could confirm this by also enabling debug in the placment api log and use teh request id do corralate the attempt to claim the allocation.

> 
> 
> Zitat von DHilsbos at performair.com:
> 
> > All;
> > 
> > Thank you for reading my previous email, despite its poor Subject.
> > 
> > I have attached a subset of the Nova Scheduler log, with debugging  
> > enabled.  I have added line numbers to the beginning, in order to  
> > facilitate referencing.
> > 
> > At line 18 it appears to select the requested server for the  
> > migration, but then at line 38, it rejects it.
> > 
> > Am I missing something here?  Is it unable to assign the resources  
> > through placement?
> > 
> > Dominic L. Hilsbos, MBA
> > Vice President – Information Technology
> > Perform Air International Inc.
> > DHilsbos at PerformAir.com
> > www.PerformAir.com
> > 
> > 
> > -----Original Message-----
> > From: melanie witt [mailto:melwittt at gmail.com]
> > Sent: Wednesday, July 28, 2021 1:24 PM
> > To: Dominic Hilsbos; openstack-discuss at lists.openstack.org
> > Subject: Re: [ops][nova]
> > 
> > On Wed, 28 Jul 2021, 20:10 +0000, <DHilsbos at performair.com> wrote:
> > > All;
> > > 
> > > I'm having a problem, though I'm not certain of the full scope.
> > > 
> > > The most obvious issue is that I can't live migrate instances.  The  
> > > server I want to send the instance to gets filtered out, though I  
> > > don't see any logs that tell me why it's being filtered out.
> > > 
> > > It might be related that all of my new instances seems to land on a  
> > > single host.
> > > 
> > > An pointers on where to look for logs for Host Filters?
> > 
> > You'll want to set [DEFAULT]debug = True in the nova.conf for your
> > nova-scheduler service and restart or HUP the service. And then the
> > logging per scheduler filter (each one will show the starting and ending
> > number of hosts) will be included at DEBUG log level in the
> > nova-scheduler log.
> > 
> > -melwitt
> 
> 
> 
>