Open Stack

On Tue, 2019-01-22 at 09:20 -0600, Matt Riedemann wrote:
> While working on the code to create instance-related data in the target 
> cell database during a cross-cell resize I noticed I wasn't copying over 
> pci_devices [1] but when looking at the PciDevice object create/save 
> methods, those aren't really what I'm looking for here. And looking at 
> the data model, there is a compute_node_id field which is the 
> compute_nodes.id primary key which won't match the target cell DB.
> 
> I am not very familiar with the PCI device manager code and data model 
> (it's my weakest area in nova lo these many years) but looking closer at 
> this, am I correct in understanding that the PciDevice object and data 
> model is really more about the actual inventory and allocations of PCI 
> devices on a given compute node and therefore it doesn't really make 
> sense to need to copy that data over to the target cell database.
> 
> During a cross-cell resize, the scheduler is going to pick a target host 
> in another cell and claim standard resources (VCPU, MEMORY_MB and 
> DISK_GB) in placement, but things like NUMA/PCI claims won't happen 
> until we do a ResourceTracker.resize_claim on the target host in the 
> target cell. In that case, it seems the things I only need to care about 
> mirroring is instance.pci_requests and instance.numa_topology, correct?
yes i belive that is correct.
the scheduler when seleciting the host in the remote numa cell will need
to run the pci passthough filter to validate that pci_request against the
destingation host. 

you are specifically doing a resize so you dont need to regenerate a new xml
on the source node before starting a live migration sice its not a live migration
but you might want to premtpvily allcoate the pci devices on the destination
if you want to prevent a race with other hosts. That said for stien it may be better
to declare that out of scope. its really not any more racy then spawning an instace
as we dont claim the device untill we get to the compute node anyway today.

the instance.numa_topology shoudl really be recalulated for the target host also.
you do not want to require the destination host to place the vm with the original
numa toptolgy from the source node. so i think you need to propagate the numa related
request which are all in the flavor/image but i dont think you need to copy 
the instance numa_topology object. its not a live migration so provided the

numa toplogy filter says the host is vlaid you are free to recalualte the numa toplogy
form scratch when it lands on the compute node based on the image and flavor values.


>  
> Since those are the user-requested (via flavor/image/port) resources 
> which will then result in PciDevice and NUMA allocations on the target host.
> 
> I'm just looking for confirmation from others that better understand the 
> data model in this area.
> 
> [1] 
> https://review.openstack.org/#/c/627892/5/nova/conductor/tasks/cross_cell_migrate.py@115
>