Open Stack

Hi,

 > Do I still understand correctly ? If yes, perfect, let's jump to my 
upgrade
 > concern.

Yes, I think. The old microversions look into only root providers and 
give up providing resources if a root provider itself doesn't have 
enough inventories for requested resources. But the new microversion 
looks into the root's descendents also and see if it can provide 
requested resources *collectively* in that tree.

The tests from [1] would help you understand this, where VCPUs come from 
the root(compute host) and SRIOV_NET_VFs from its grandchild.

[1] 
https://review.openstack.org/#/c/565487/15/nova/tests/functional/api/openstack/placement/gabbits/allocation-candidates.yaml@362

 > In that situation, say for example with VGPU inventories, that would mean
 > that the compute node would stop reporting inventories for its root 
RP, but
 > would rather report inventories for at least one single child RP.
 > In that model, do we reconcile the allocations that were already made
 > against the "root RP" inventory ?

It would be nice to see Eric and Jay comment on this,
but if I'm not mistaken, when the virt driver stops reporting 
inventories for its root RP, placement would try to delete that 
inventory inside and raise InventoryInUse exception if any allocations 
still exist on that resource.

```
update_from_provider_tree() (nova/compute/resource_tracker.py)
   + _set_inventory_for_provider() (nova/scheduler/client/report.py)
       + put() - PUT /resource_providers/<rp_uuid>/inventories with new 
inventories (scheduler/client/report.py)
           + set_inventories() (placement/handler/inventory.py)
               + _set_inventory() (placement/objects/resource_proveider.py)
                   + _delete_inventory_from_provider() 
(placement/objects/resource_proveider.py)
                       -> raise exception.InventoryInUse
```

So we need some trick something like deleting VGPU allocations before 
upgrading and set the allocation again for the created new child after 
upgrading?

On 2018/05/28 23:18, Sylvain Bauza wrote:
> Hi,
> 
> I already told about that in a separate thread, but let's put it here too
> for more visibility.
> 
> tl;dr: I suspect existing allocations are being lost when we upgrade a
> compute service from Queens to Rocky, if those allocations are made against
> inventories that are now provided by a child Resource Provider.
> 
> 
> I started reviewing https://review.openstack.org/#/c/565487/ and bottom
> patches to understand the logic with querying nested resource providers.
>>From what I understand, the scheduler will query Placement using the same
> query but will get (thanks to a new microversion) not only allocation
> candidates that are root resource providers but also any possible child.
> 
> If so, that's great as in a rolling upgrade scenario with mixed computes
> (both Queens and Rocky), we will still continue to return both old RPs and
> new child RPs if they both support the same resource classes ask.
> Accordingly, allocations done by the scheduler will be made against the
> corresponding Resource Provider, whether it's a root RP (old way) or a
> child RP (new way).
> 
> Do I still understand correctly ? If yes, perfect, let's jump to my upgrade
> concern.
> Now, consider the Queens->Rocky compute upgrade. If I'm an operator and I
> start deploying Rocky on one compute node, it will provide to Placement API
> new inventories that are possibly nested.
> In that situation, say for example with VGPU inventories, that would mean
> that the compute node would stop reporting inventories for its root RP, but
> would rather report inventories for at least one single child RP.
> In that model, do we reconcile the allocations that were already made
> against the "root RP" inventory ? I don't think so, hence my question here.
> 
> Thanks,
> -Sylvain
> 
> 
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

-- 
Tetsuro Nakamura <nakamura.tetsuro at lab.ntt.co.jp>
NTT Network Service Systems Laboratories
TEL:0422 59 6914(National)/+81 422 59 6914(International)
3-9-11, Midori-Cho Musashino-Shi, Tokyo 180-8585 Japan