Open Stack

On 09/16/2018 09:28 PM, Naichuan Sun wrote:
> Hi, Sylvain,
> 
> In truth I’m worrying about the old root rp which include the vgpu 
> inventory. There is no field in the inventory which can display which 
> GPU/GPUG it belong to, right? Anyway,  will discuss it after you come back.

As Sylvain mentions below, you will need to have some mechanism in the 
XenAPI virt driver which creates child resource providers under the 
existing root provider (which is the compute node resource provider). 
You will need to have the virt driver persist the mapping between your 
internal physical GPU group name and the UUID of the resource provider 
record that the virt driver creates for that PGPU group.

So, for example, let's say you have two PGPU groups on the host. They 
are named PGPU_A and PGPU_B. The XenAPI virt driver will need to ask the 
ProviderTree object it receives in the update_provider_tree() virt 
driver method whether there is a resource provider named "PGPU_A" in the 
tree. If not, the virt driver needs to create a new child resource 
provider with the name "PGPU_A" with a parent provider pointing to the 
root compute node provider. The ProviderTree.new_child() method is used 
to create new child providers:

https://github.com/openstack/nova/blob/82270cc261f6c1d9d2cc386f1fb445dd66023f75/nova/compute/provider_tree.py#L411

Hope that makes sense,
-jay

> Thank very much.
> 
> BR.
> 
> Naichuan Sun
> 
> *From:*Sylvain Bauza [mailto:sbauza at redhat.com]
> *Sent:* Friday, September 14, 2018 9:34 PM
> *To:* OpenStack Development Mailing List (not for usage questions) 
> <openstack-dev at lists.openstack.org>
> *Subject:* Re: [openstack-dev] About microversion setting to enable 
> nested resource provider
> 
> Le jeu. 13 sept. 2018 à 19:29, Naichuan Sun <naichuan.sun at citrix.com 
> <mailto:naichuan.sun at citrix.com>> a écrit :
> 
>     Hi, Sylvain,
> 
>     Thank you very much for the information. It is pity that I can’t
>     attend the meeting.
> 
>     I have a concern about reshaper in multi-type vgpu support.
> 
>     In the old vgpu support, we only have one vgpu inventory in root
>     resource provider, which means we only support one vgpu type. When
>     do reshape, placement will send allocations(which include just one
>     vgpu resource allocation information) to the driver, if the host
>     have more than one pgpu/pgpug(which support different vgpu type),
>     how do we know which pgpu/pgpug own the allocation information? Do
>     we need to communicate with hypervisor the confirm that?
> 
> The reshape will actually move the existing allocations for a VGPU 
> resource class to the inventory for this class that is on the child 
> resource provider now with the reshape.
> 
> Since we agreed on keeping consistent naming, there is no need to guess 
> which is which. That said, you raise a point that was discussed during 
> the PTG and we all agreed there was an upgrade impact as multiple vGPUs 
> shouldn't be allowed until the reshape is done.
> 
> Accordingly, see my spec I reproposed for Stein which describes the 
> upgrade impact https://review.openstack.org/#/c/602474/
> 
> Since I'm at the PTG, we have huge time difference between you and me, 
> but we can discuss on that point next week when I'm back (my mornings 
> match then your afternoons)
> 
> -Sylvain
> 
>     Thank you very much.
> 
>     BR.
> 
>     Naichuan Sun
> 
>     *From:*Sylvain Bauza [mailto:sbauza at redhat.com
>     <mailto:sbauza at redhat.com>]
>     *Sent:* Thursday, September 13, 2018 11:47 PM
>     *To:* OpenStack Development Mailing List (not for usage questions)
>     <openstack-dev at lists.openstack.org
>     <mailto:openstack-dev at lists.openstack.org>>
>     *Subject:* Re: [openstack-dev] About microversion setting to enable
>     nested resource provider
> 
>     Hey Naichuan,
> 
>     FWIW, we discussed on the missing pieces for nested resource
>     providers. See the (currently-in-use) etherpad
>     https://etherpad.openstack.org/p/nova-ptg-stein and lookup for
>     "closing the gap on nested resource providers" (L144 while I speak)
> 
>     The fact that we are not able to schedule yet is a critical piece
>     that we said we're going to work on it as soon as we can.
> 
>     -Sylvain
> 
>     On Thu, Sep 13, 2018 at 9:14 AM, Eric Fried <openstack at fried.cc
>     <mailto:openstack at fried.cc>> wrote:
> 
>         There's a patch series in progress for this:
> 
>         https://review.openstack.org/#/q/topic:use-nested-allocation-candidates
> 
>         It needs some TLC. I'm sure gibi and tetsuro would welcome some
>         help...
> 
>         efried
> 
> 
>         On 09/13/2018 08:31 AM, Naichuan Sun wrote:
>          > Thank you very much, Jay.
>          > Is there somewhere I could set microversion(some configure
>         file?), Or just modify the source code to set it?
>          >
>          > BR.
>          > Naichuan Sun
>          >
>          > -----Original Message-----
>          > From: Jay Pipes [mailto:jaypipes at gmail.com
>         <mailto:jaypipes at gmail.com>]
>          > Sent: Thursday, September 13, 2018 9:19 PM
>          > To: Naichuan Sun <naichuan.sun at citrix.com
>         <mailto:naichuan.sun at citrix.com>>; OpenStack Development Mailing
>         List (not for usage questions)
>         <openstack-dev at lists.openstack.org
>         <mailto:openstack-dev at lists.openstack.org>>
>          > Cc: melanie witt <melwittt at gmail.com
>         <mailto:melwittt at gmail.com>>; efried at us.ibm.com
>         <mailto:efried at us.ibm.com>; Sylvain Bauza <sbauza at redhat.com
>         <mailto:sbauza at redhat.com>>
>          > Subject: Re: About microversion setting to enable nested
>         resource provider
>          >
>          > On 09/13/2018 06:39 AM, Naichuan Sun wrote:
>          >> Hi, guys,
>          >>
>          >> Looks n-rp is disabled by default because microversion
>         matches 1.29 :
>          >>
>         https://github.com/openstack/nova/blob/master/nova/api/openstack/place
>          >> ment/handlers/allocation_candidate.py#L252
>          >>
>          >> Anyone know how to set the microversion to enable n-rp in
>         placement?
>          >
>          > It is the client which must send the 1.29+ placement API
>         microversion header to indicate to the placement API server that
>         the client wants to receive nested provider information in the
>         allocation candidates response.
>          >
>          > Currently, nova-scheduler calls the scheduler reportclient's
>          > get_allocation_candidates() method:
>          >
>          >
>         https://github.com/openstack/nova/blob/0ba34a818414823eda5e693dc2127a534410b5df/nova/scheduler/manager.py#L138
>          >
>          > The scheduler reportclient's get_allocation_candidates()
>         method currently passes the 1.25 placement API microversion header:
>          >
>          >
>         https://github.com/openstack/nova/blob/0ba34a818414823eda5e693dc2127a534410b5df/nova/scheduler/client/report.py#L353
>          >
>          >
>         https://github.com/openstack/nova/blob/0ba34a818414823eda5e693dc2127a534410b5df/nova/scheduler/client/report.py#L53
>          >
>          > In order to get the nested information returned in the
>         allocation candidates response, that would need to be upped to 1.29.
>          >
>          > Best,
>          > -jay
> 
>          >
>         __________________________________________________________________________
>          > OpenStack Development Mailing List (not for usage questions)
>          > Unsubscribe:
>         OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>         <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>          > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>          >
> 
>         __________________________________________________________________________
>         OpenStack Development Mailing List (not for usage questions)
>         Unsubscribe:
>         OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>         <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>         http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
>     __________________________________________________________________________
>     OpenStack Development Mailing List (not for usage questions)
>     Unsubscribe:
>     OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>