Open Stack

On Mon, Sep 17, 2018 at 6:43 AM, Jay Pipes <jaypipes at gmail.com> wrote:

> On 09/16/2018 09:28 PM, Naichuan Sun wrote:
>
>> Hi, Sylvain,
>>
>> In truth I’m worrying about the old root rp which include the vgpu
>> inventory. There is no field in the inventory which can display which
>> GPU/GPUG it belong to, right? Anyway,  will discuss it after you come back.
>>
>
> As Sylvain mentions below, you will need to have some mechanism in the
> XenAPI virt driver which creates child resource providers under the
> existing root provider (which is the compute node resource provider). You
> will need to have the virt driver persist the mapping between your internal
> physical GPU group name and the UUID of the resource provider record that
> the virt driver creates for that PGPU group.
>

AFAICT, we even don't need to persist the mapping. As we only support one
GPU type (or group for Xen) in Rocky, you just have to know what was the
original type for Stein and then just look at the related resource provider;
That's why i wrote an upgrade impact section in my multiple-types spec (see
below) saying that in Stein, you need to make sure you only accept one type
until the reshape is fully done.

-Sylvain


> So, for example, let's say you have two PGPU groups on the host. They are
> named PGPU_A and PGPU_B. The XenAPI virt driver will need to ask the
> ProviderTree object it receives in the update_provider_tree() virt driver
> method whether there is a resource provider named "PGPU_A" in the tree. If
> not, the virt driver needs to create a new child resource provider with the
> name "PGPU_A" with a parent provider pointing to the root compute node
> provider. The ProviderTree.new_child() method is used to create new child
> providers:
>
> https://github.com/openstack/nova/blob/82270cc261f6c1d9d2cc3
> 86f1fb445dd66023f75/nova/compute/provider_tree.py#L411
>
> Hope that makes sense,
> -jay
>
> Thank very much.
>>
>> BR.
>>
>> Naichuan Sun
>>
>> *From:*Sylvain Bauza [mailto:sbauza at redhat.com]
>> *Sent:* Friday, September 14, 2018 9:34 PM
>> *To:* OpenStack Development Mailing List (not for usage questions) <
>> openstack-dev at lists.openstack.org>
>> *Subject:* Re: [openstack-dev] About microversion setting to enable
>> nested resource provider
>>
>> Le jeu. 13 sept. 2018 à 19:29, Naichuan Sun <naichuan.sun at citrix.com
>> <mailto:naichuan.sun at citrix.com>> a écrit :
>>
>>     Hi, Sylvain,
>>
>>     Thank you very much for the information. It is pity that I can’t
>>     attend the meeting.
>>
>>     I have a concern about reshaper in multi-type vgpu support.
>>
>>     In the old vgpu support, we only have one vgpu inventory in root
>>     resource provider, which means we only support one vgpu type. When
>>     do reshape, placement will send allocations(which include just one
>>     vgpu resource allocation information) to the driver, if the host
>>     have more than one pgpu/pgpug(which support different vgpu type),
>>     how do we know which pgpu/pgpug own the allocation information? Do
>>     we need to communicate with hypervisor the confirm that?
>>
>> The reshape will actually move the existing allocations for a VGPU
>> resource class to the inventory for this class that is on the child
>> resource provider now with the reshape.
>>
>> Since we agreed on keeping consistent naming, there is no need to guess
>> which is which. That said, you raise a point that was discussed during the
>> PTG and we all agreed there was an upgrade impact as multiple vGPUs
>> shouldn't be allowed until the reshape is done.
>>
>> Accordingly, see my spec I reproposed for Stein which describes the
>> upgrade impact https://review.openstack.org/#/c/602474/
>>
>> Since I'm at the PTG, we have huge time difference between you and me,
>> but we can discuss on that point next week when I'm back (my mornings match
>> then your afternoons)
>>
>> -Sylvain
>>
>>     Thank you very much.
>>
>>     BR.
>>
>>     Naichuan Sun
>>
>>     *From:*Sylvain Bauza [mailto:sbauza at redhat.com
>>     <mailto:sbauza at redhat.com>]
>>     *Sent:* Thursday, September 13, 2018 11:47 PM
>>     *To:* OpenStack Development Mailing List (not for usage questions)
>>     <openstack-dev at lists.openstack.org
>>     <mailto:openstack-dev at lists.openstack.org>>
>>     *Subject:* Re: [openstack-dev] About microversion setting to enable
>>     nested resource provider
>>
>>     Hey Naichuan,
>>
>>     FWIW, we discussed on the missing pieces for nested resource
>>     providers. See the (currently-in-use) etherpad
>>     https://etherpad.openstack.org/p/nova-ptg-stein and lookup for
>>     "closing the gap on nested resource providers" (L144 while I speak)
>>
>>     The fact that we are not able to schedule yet is a critical piece
>>     that we said we're going to work on it as soon as we can.
>>
>>     -Sylvain
>>
>>     On Thu, Sep 13, 2018 at 9:14 AM, Eric Fried <openstack at fried.cc
>>     <mailto:openstack at fried.cc>> wrote:
>>
>>         There's a patch series in progress for this:
>>
>>         https://review.openstack.org/#/q/topic:use-nested-allocation
>> -candidates
>>
>>         It needs some TLC. I'm sure gibi and tetsuro would welcome some
>>         help...
>>
>>         efried
>>
>>
>>         On 09/13/2018 08:31 AM, Naichuan Sun wrote:
>>          > Thank you very much, Jay.
>>          > Is there somewhere I could set microversion(some configure
>>         file?), Or just modify the source code to set it?
>>          >
>>          > BR.
>>          > Naichuan Sun
>>          >
>>          > -----Original Message-----
>>          > From: Jay Pipes [mailto:jaypipes at gmail.com
>>         <mailto:jaypipes at gmail.com>]
>>          > Sent: Thursday, September 13, 2018 9:19 PM
>>          > To: Naichuan Sun <naichuan.sun at citrix.com
>>         <mailto:naichuan.sun at citrix.com>>; OpenStack Development Mailing
>>         List (not for usage questions)
>>         <openstack-dev at lists.openstack.org
>>         <mailto:openstack-dev at lists.openstack.org>>
>>          > Cc: melanie witt <melwittt at gmail.com
>>         <mailto:melwittt at gmail.com>>; efried at us.ibm.com
>>         <mailto:efried at us.ibm.com>; Sylvain Bauza <sbauza at redhat.com
>>         <mailto:sbauza at redhat.com>>
>>
>>          > Subject: Re: About microversion setting to enable nested
>>         resource provider
>>          >
>>          > On 09/13/2018 06:39 AM, Naichuan Sun wrote:
>>          >> Hi, guys,
>>          >>
>>          >> Looks n-rp is disabled by default because microversion
>>         matches 1.29 :
>>          >>
>>         https://github.com/openstack/nova/blob/master/nova/api/opens
>> tack/place
>>          >> ment/handlers/allocation_candidate.py#L252
>>          >>
>>          >> Anyone know how to set the microversion to enable n-rp in
>>         placement?
>>          >
>>          > It is the client which must send the 1.29+ placement API
>>         microversion header to indicate to the placement API server that
>>         the client wants to receive nested provider information in the
>>         allocation candidates response.
>>          >
>>          > Currently, nova-scheduler calls the scheduler reportclient's
>>          > get_allocation_candidates() method:
>>          >
>>          >
>>         https://github.com/openstack/nova/blob/0ba34a818414823eda5e6
>> 93dc2127a534410b5df/nova/scheduler/manager.py#L138
>>          >
>>          > The scheduler reportclient's get_allocation_candidates()
>>         method currently passes the 1.25 placement API microversion
>> header:
>>          >
>>          >
>>         https://github.com/openstack/nova/blob/0ba34a818414823eda5e6
>> 93dc2127a534410b5df/nova/scheduler/client/report.py#L353
>>          >
>>          >
>>         https://github.com/openstack/nova/blob/0ba34a818414823eda5e6
>> 93dc2127a534410b5df/nova/scheduler/client/report.py#L53
>>          >
>>          > In order to get the nested information returned in the
>>         allocation candidates response, that would need to be upped to
>> 1.29.
>>          >
>>          > Best,
>>          > -jay
>>
>>          >
>>         ____________________________________________________________
>> ______________
>>          > OpenStack Development Mailing List (not for usage questions)
>>          > Unsubscribe:
>>         OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>         <http://OpenStack-dev-request@lists.openstack.org?subject:un
>> subscribe>
>>          > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstac
>> k-dev
>>          >
>>
>>         ____________________________________________________________
>> ______________
>>         OpenStack Development Mailing List (not for usage questions)
>>         Unsubscribe:
>>         OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>         <http://OpenStack-dev-request@lists.openstack.org?subject:un
>> subscribe>
>>         http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>     ____________________________________________________________
>> ______________
>>     OpenStack Development Mailing List (not for usage questions)
>>     Unsubscribe:
>>     OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> ____________________________________________________________
>> ______________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180917/04ec86a8/attachment.html>