<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Tue, May 29, 2018 at 3:08 AM, TETSURO NAKAMURA <span dir="ltr"><<a href="mailto:nakamura.tetsuro@lab.ntt.co.jp" target="_blank">nakamura.tetsuro@lab.ntt.co.jp</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<span class=""><br>
<br>
> Do I still understand correctly ? If yes, perfect, let's jump to my upgrade<br>
> concern.<br>
<br></span>
Yes, I think. The old microversions look into only root providers and give up providing resources if a root provider itself doesn't have enough inventories for requested resources. But the new microversion looks into the root's descendents also and see if it can provide requested resources *collectively* in that tree.<br>
<br>
The tests from [1] would help you understand this, where VCPUs come from the root(compute host) and SRIOV_NET_VFs from its grandchild.<br>
<br>
[1] <a href="https://review.openstack.org/#/c/565487/15/nova/tests/functional/api/openstack/placement/gabbits/allocation-candidates.yaml@362" rel="noreferrer" target="_blank">https://review.openstack.org/#<wbr>/c/565487/15/nova/tests/functi<wbr>onal/api/openstack/placement/<wbr>gabbits/allocation-candidates.<wbr>yaml@362</a><span class=""><br>
<br></span></blockquote><div><br></div><div>Yeah I already saw those tests, but I wanted to make sure I was correctly understanding.<br> <br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">
> In that situation, say for example with VGPU inventories, that would mean<br>
> that the compute node would stop reporting inventories for its root RP, but<br>
> would rather report inventories for at least one single child RP.<br>
> In that model, do we reconcile the allocations that were already made<br>
> against the "root RP" inventory ?<br>
<br></span>
It would be nice to see Eric and Jay comment on this,<br>
but if I'm not mistaken, when the virt driver stops reporting inventories for its root RP, placement would try to delete that inventory inside and raise InventoryInUse exception if any allocations still exist on that resource.<br>
<br>
```<br>
update_from_provider_tree() (nova/compute/resource_tracker<wbr>.py)<br>
+ _set_inventory_for_provider() (nova/scheduler/client/report.<wbr>py)<br>
+ put() - PUT /resource_providers/<rp_uuid>/<wbr>inventories with new inventories (scheduler/client/report.py)<br>
+ set_inventories() (placement/handler/inventory.p<wbr>y)<br>
+ _set_inventory() (placement/objects/resource_pr<wbr>oveider.py)<br>
+ _delete_inventory_from_provide<wbr>r() (placement/objects/resource_pr<wbr>oveider.py)<br>
-> raise exception.InventoryInUse<br>
```<br>
<br>
So we need some trick something like deleting VGPU allocations before upgrading and set the allocation again for the created new child after upgrading?<div><div class="h5"><br></div></div></blockquote><div><br></div><div>I wonder if we should keep the existing inventory in the root RP, and somehow just reserve the left resources (so Placement wouldn't pass that root RP for queries, but would still have allocations). But then, where and how to do this ? By the resource tracker ?<br><br></div><div>-Sylvain<br><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5">
<br>
On 2018/05/28 23:18, Sylvain Bauza wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5">
Hi,<br>
<br>
I already told about that in a separate thread, but let's put it here too<br>
for more visibility.<br>
<br>
tl;dr: I suspect existing allocations are being lost when we upgrade a<br>
compute service from Queens to Rocky, if those allocations are made against<br>
inventories that are now provided by a child Resource Provider.<br>
<br>
<br>
I started reviewing <a href="https://review.openstack.org/#/c/565487/" rel="noreferrer" target="_blank">https://review.openstack.org/#<wbr>/c/565487/</a> and bottom<br>
patches to understand the logic with querying nested resource providers.<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
>From what I understand, the scheduler will query Placement using the same<br>
</blockquote>
query but will get (thanks to a new microversion) not only allocation<br>
candidates that are root resource providers but also any possible child.<br>
<br>
If so, that's great as in a rolling upgrade scenario with mixed computes<br>
(both Queens and Rocky), we will still continue to return both old RPs and<br>
new child RPs if they both support the same resource classes ask.<br>
Accordingly, allocations done by the scheduler will be made against the<br>
corresponding Resource Provider, whether it's a root RP (old way) or a<br>
child RP (new way).<br>
<br>
Do I still understand correctly ? If yes, perfect, let's jump to my upgrade<br>
concern.<br>
Now, consider the Queens->Rocky compute upgrade. If I'm an operator and I<br>
start deploying Rocky on one compute node, it will provide to Placement API<br>
new inventories that are possibly nested.<br>
In that situation, say for example with VGPU inventories, that would mean<br>
that the compute node would stop reporting inventories for its root RP, but<br>
would rather report inventories for at least one single child RP.<br>
In that model, do we reconcile the allocations that were already made<br>
against the "root RP" inventory ? I don't think so, hence my question here.<br>
<br>
Thanks,<br>
-Sylvain<br>
<br>
<br>
<br></div></div>
______________________________<wbr>______________________________<wbr>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.op<wbr>enstack.org?subject:unsubscrib<wbr>e</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi<wbr>-bin/mailman/listinfo/openstac<wbr>k-dev</a><br>
<br><span class="HOEnZb"><font color="#888888">
</font></span></blockquote><span class="HOEnZb"><font color="#888888">
<br>
-- <br>
Tetsuro Nakamura <<a href="mailto:nakamura.tetsuro@lab.ntt.co.jp" target="_blank">nakamura.tetsuro@lab.ntt.co.j<wbr>p</a>><br>
NTT Network Service Systems Laboratories<br>
TEL:0422 59 6914(National)/+81 422 59 6914(International)<br>
3-9-11, Midori-Cho Musashino-Shi, Tokyo 180-8585 Japan<br>
<br>
<br>
</font></span></blockquote></div><br></div></div>