Open Stack

On Tue, 19 Jul 2016 12:40:50 -0700
Ed Leafe <ed at leafe.com> wrote:

> > It can identified 3 levels of FPGA resources, which can be nested one
> > on the others:
> >
> > 1. Whole FPGA. If used discrete FPGA, than even today it might be pass
> >   through to the VM.
> Can you explain why this would ever be useful? IOW, what can a VM do
> with an entire FPGA?

Private cloud, development purposes. It could be treated as FPGA-aaS.
Same goes for the unoccupied slots. Of course, because reprogramming
affects real, not virtualized hardware, there are security concerns
for allowing users to do that in public clouds for example.

The other reason, which is much more significant, there could be IPs
so big, that will take most of FPGA - so reconfiguration is needed
(although reconfiguration is out of scope for Nova) and we don't want
to wipe out other accelerator which might be currently in use. That's
why slots here also are treated as dynamic resources.

> > 3. Accelerator in region/FPGA. If there is an accelerator programmed
> >   in the slot, it is possible, that such accelerator provides us with
> >   Virtual Functions (similar to the SR-IOV), than every available VF
> >   can be treated as a resource.
>
> This is my understanding of what would be consumable: the slot / VF,
> which the VM could take advantage of.

Yes. That's the obvious scenario.

> > 4. It might be also necessary to track every VF individually, although
> >   I didn't assumed it will be needed, nevertheless with nested
> >   resources it should be easy to handle it.
>
> I’m still not seeing the need for nesting. If you have a single FPGA
> with 8 slots, when you program the slots with accelerators, you now
> have 8 consumable resources. The fact that they came from a
> particular FPGA unit doesn’t seem relevant from a scheduling
> perspective.

Unless you have one FPGA with 8 slots, which can become FPGA with 4
slots. From scheduling perspective you have to know, which FPGA
resources can be reconfigured, and which not, isn't it? Also, AFAIRC
to provide VM with VF, there is a need for providing libvirt with
address of such VF, right? That's why I've putted this last point.

The whole idea of getting FPGA as resource is its ability to swap
resources on demand. So it can be thought of as several available
hardware (means - accelerators, consumable by VMs) which most of the
time are not programmed in certain moment.

So, let's assume, that we have two hosts: HostA and HostB with FPGA
capable to provide 2 accelerators which exclusively use entire chip
(lets call them AX1 and AX2), and one other, which can use one of the
2 possible slots (AY). So, the situation is we have 3 possible
accelerators to use, and in worst case scenario only two available
places where they can be places.

Initially, there is no accelerator in use, cloud administrator define
all the IPs he have available (somehow - this part isn't defined yet -
but lets assume it is in place)

Now, user requests VM with certain flavor/image with AX1 and scheduler
knows, that it will fit into HostA and HostB, so HostA is chosen, FPGA
magically™ is prepared to hold AX1 accelerator and VM was started. Now
we have resource tree HostA FPGA->slot->AX1 and HostB FPGA.

Next, user requests another VM with AY accelerator, scheduler now
should know, that the only available option is HostB, so again magic
is happening, and there is a resource tree:

HostA FPGA
 +- slot1
     +- AX1

HostB FPGA
 +- slot1
     +- AY
 +- slot2


Now, what should happen if user remove VM with AY accelerator and
request another VM with AX2?

-- 
Cheers,
Roman Dobosz