Open Stack

Roman, great thread, thanks for posting! Comment inline :)

On 07/19/2016 02:03 PM, Roman Dobosz wrote:
<snip>
> It can identified 3 levels of FPGA resources, which can be nested one
> on the others:
>
> 1. Whole FPGA. If used discrete FPGA, than even today it might be pass
>    through to the VM.
>
> 2. Region in FPGA. Some of the FPGA models can be divided into regions
>    or slots. Also, for some model it is possible to (re)program such
>    region individually - in this case there is a possibility to pass
>    entire slot to the VM, so that it might be possible to reprogram
>    such slot, and utilize the algorithm within the VM.
>
> 3. Accelerator in region/FPGA. If there is an accelerator programmed
>    in the slot, it is possible, that such accelerator provides us with
>    Virtual Functions (similar to the SR-IOV), than every available VF
>    can be treated as a resource.
>
> 4. It might be also necessary to track every VF individually, although
>    I didn't assumed it will be needed, nevertheless with nested
>    resources it should be easy to handle it.
>
> Correlation between such resources are a bit different from NUMA -
> while in NUMA case there is a possibility to either schedule a VM with
> some memory specified, or request memory within NUMA cell, in FPGA if
> there is slot taken, or accelerator already programmed and used, there
> is no way to offer FPGA as a whole to the tenant, until all
> accelerators and slots are free.
>
> I've followed Jay idea about nested resources and having in mind
> blueprint[2] regarding dynamic resources I've prepared how it fit in.
>
<snip>
>
> To get id of resource of type acceleratorX to allocate 8 VF:
>
>
> SELECT rp.id
> FROM resource_providers rp
> LEFT JOIN allocations al ON al.resource_provider_id = rp.id
> LEFT JOIN inventories iv ON iv.resource_provider_id = rp.id
> WHERE al.resource_class_id = 1668
> AND (iv.total - COALESCE(al.used, 0)) >= 8;

Right idea, yes, but you would need to INNER JOIN inventories and LEFT 
JOIN from the winnowed set of inventory records to a grouped projection 
of allocations. :)

The SQL would be this:

SELECT rp.id
FROM resource_providers rp
INNER JOIN inventories iv
ON rp.id = iv.resource_provider_id
AND iv.resource_class_id = 1688
LEFT JOIN (
   SELECT resource_provider_id, SUM(used) as used
   FROM allocations
   WHERE resource_class_id = 1688
   GROUP BY resource_provider_id
) AS al
ON iv.resource_provider_id = al.id
WHERE (iv.total - COALESCE(al.used, 0)) >= 8;

The other SQL queries you listed had a couple errors, but the ideas were 
mostly sound. I'll include the FPGA use cases when I write up the nested 
resource providers spec proposal.

The only thing I'd say is that I was envisioning the dynamic resource 
classes for FPGAs to be the resource context to an already-flashed 
algorithm, not to the FPGA root device (or a region even). But, who 
knows, perhaps we can work something out. More discussion on the spec...

Best,
-jay