Open Stack

On Tue, May 29, 2018 at 3:08 AM, TETSURO NAKAMURA <
nakamura.tetsuro at lab.ntt.co.jp> wrote:

> Hi,
>
> > Do I still understand correctly ? If yes, perfect, let's jump to my
> upgrade
> > concern.
>
> Yes, I think. The old microversions look into only root providers and give
> up providing resources if a root provider itself doesn't have enough
> inventories for requested resources. But the new microversion looks into
> the root's descendents also and see if it can provide requested resources
> *collectively* in that tree.
>
> The tests from [1] would help you understand this, where VCPUs come from
> the root(compute host) and SRIOV_NET_VFs from its grandchild.
>
> [1] https://review.openstack.org/#/c/565487/15/nova/tests/functi
> onal/api/openstack/placement/gabbits/allocation-candidates.yaml at 362
>
>
Yeah I already saw those tests, but I wanted to make sure I was correctly
understanding.


> > In that situation, say for example with VGPU inventories, that would mean
> > that the compute node would stop reporting inventories for its root RP,
> but
> > would rather report inventories for at least one single child RP.
> > In that model, do we reconcile the allocations that were already made
> > against the "root RP" inventory ?
>
> It would be nice to see Eric and Jay comment on this,
> but if I'm not mistaken, when the virt driver stops reporting inventories
> for its root RP, placement would try to delete that inventory inside and
> raise InventoryInUse exception if any allocations still exist on that
> resource.
>
> ```
> update_from_provider_tree() (nova/compute/resource_tracker.py)
>   + _set_inventory_for_provider() (nova/scheduler/client/report.py)
>       + put() - PUT /resource_providers/<rp_uuid>/inventories with new
> inventories (scheduler/client/report.py)
>           + set_inventories() (placement/handler/inventory.py)
>               + _set_inventory() (placement/objects/resource_proveider.py)
>                   + _delete_inventory_from_provider()
> (placement/objects/resource_proveider.py)
>                       -> raise exception.InventoryInUse
> ```
>
> So we need some trick something like deleting VGPU allocations before
> upgrading and set the allocation again for the created new child after
> upgrading?
>
>
I wonder if we should keep the existing inventory in the root RP, and
somehow just reserve the left resources (so Placement wouldn't pass that
root RP for queries, but would still have allocations). But then, where and
how to do this ? By the resource tracker ?

-Sylvain


> On 2018/05/28 23:18, Sylvain Bauza wrote:
>
>> Hi,
>>
>> I already told about that in a separate thread, but let's put it here too
>> for more visibility.
>>
>> tl;dr: I suspect existing allocations are being lost when we upgrade a
>> compute service from Queens to Rocky, if those allocations are made
>> against
>> inventories that are now provided by a child Resource Provider.
>>
>>
>> I started reviewing https://review.openstack.org/#/c/565487/ and bottom
>> patches to understand the logic with querying nested resource providers.
>>
>>> From what I understand, the scheduler will query Placement using the same
>>>
>> query but will get (thanks to a new microversion) not only allocation
>> candidates that are root resource providers but also any possible child.
>>
>> If so, that's great as in a rolling upgrade scenario with mixed computes
>> (both Queens and Rocky), we will still continue to return both old RPs and
>> new child RPs if they both support the same resource classes ask.
>> Accordingly, allocations done by the scheduler will be made against the
>> corresponding Resource Provider, whether it's a root RP (old way) or a
>> child RP (new way).
>>
>> Do I still understand correctly ? If yes, perfect, let's jump to my
>> upgrade
>> concern.
>> Now, consider the Queens->Rocky compute upgrade. If I'm an operator and I
>> start deploying Rocky on one compute node, it will provide to Placement
>> API
>> new inventories that are possibly nested.
>> In that situation, say for example with VGPU inventories, that would mean
>> that the compute node would stop reporting inventories for its root RP,
>> but
>> would rather report inventories for at least one single child RP.
>> In that model, do we reconcile the allocations that were already made
>> against the "root RP" inventory ? I don't think so, hence my question
>> here.
>>
>> Thanks,
>> -Sylvain
>>
>>
>>
>> ____________________________________________________________
>> ______________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
> --
> Tetsuro Nakamura <nakamura.tetsuro at lab.ntt.co.jp>
> NTT Network Service Systems Laboratories
> TEL:0422 59 6914(National)/+81 422 59 6914(International)
> 3-9-11, Midori-Cho Musashino-Shi, Tokyo 180-8585 Japan
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180529/04f5527e/attachment.html>