Open Stack

On Thu, 28 Jul 2016 10:50:08 -0400
Jay Pipes <jaypipes at gmail.com> wrote:

> Roman, great thread, thanks for posting! Comment inline :)

Thanks!

> <snip>
> > It can identified 3 levels of FPGA resources, which can be nested one
> > on the others:
> >
> > 1. Whole FPGA. If used discrete FPGA, than even today it might be pass
> >    through to the VM.
> >
> > 2. Region in FPGA. Some of the FPGA models can be divided into regions
> >    or slots. Also, for some model it is possible to (re)program such
> >    region individually - in this case there is a possibility to pass
> >    entire slot to the VM, so that it might be possible to reprogram
> >    such slot, and utilize the algorithm within the VM.
> >
> > 3. Accelerator in region/FPGA. If there is an accelerator programmed
> >    in the slot, it is possible, that such accelerator provides us with
> >    Virtual Functions (similar to the SR-IOV), than every available VF
> >    can be treated as a resource.
> >
> > 4. It might be also necessary to track every VF individually, although
> >    I didn't assumed it will be needed, nevertheless with nested
> >    resources it should be easy to handle it.
> >
> > Correlation between such resources are a bit different from NUMA -
> > while in NUMA case there is a possibility to either schedule a VM with
> > some memory specified, or request memory within NUMA cell, in FPGA if
> > there is slot taken, or accelerator already programmed and used, there
> > is no way to offer FPGA as a whole to the tenant, until all
> > accelerators and slots are free.
> >
> > I've followed Jay idea about nested resources and having in mind
> > blueprint[2] regarding dynamic resources I've prepared how it fit in.
> >
> <snip>
> >
> > To get id of resource of type acceleratorX to allocate 8 VF:
> >
> >
> > SELECT rp.id
> > FROM resource_providers rp
> > LEFT JOIN allocations al ON al.resource_provider_id = rp.id
> > LEFT JOIN inventories iv ON iv.resource_provider_id = rp.id
> > WHERE al.resource_class_id = 1668
> > AND (iv.total - COALESCE(al.used, 0)) >= 8;
> 
> Right idea, yes, but you would need to INNER JOIN inventories and LEFT 
> JOIN from the winnowed set of inventory records to a grouped projection 
> of allocations. :)
> 
> The SQL would be this:
> 
> SELECT rp.id
> FROM resource_providers rp
> INNER JOIN inventories iv
> ON rp.id = iv.resource_provider_id
> AND iv.resource_class_id = 1688
> LEFT JOIN (
>    SELECT resource_provider_id, SUM(used) as used
>    FROM allocations
>    WHERE resource_class_id = 1688
>    GROUP BY resource_provider_id
> ) AS al
> ON iv.resource_provider_id = al.id
> WHERE (iv.total - COALESCE(al.used, 0)) >= 8;

Hm. I'm getting same results using the both queries. Certainly, I can't
see something obvious here, and for sure I'm no sql expert :)

> The other SQL queries you listed had a couple errors, but the ideas were 
> mostly sound. I'll include the FPGA use cases when I write up the nested 
> resource providers spec proposal.

Great, thank you!

> The only thing I'd say is that I was envisioning the dynamic resource 
> classes for FPGAs to be the resource context to an already-flashed 
> algorithm, not to the FPGA root device (or a region even). But, who 
> knows, perhaps we can work something out. More discussion on the spec...

For sure, we can start from defining basic case, and expand it if
needed.

-- 
Cheers,
Roman Dobosz