Open Stack

On Jul 21, 2016 5:12 AM, "Daniel P. Berrange" <berrange at redhat.com> wrote:
>
> On Thu, Jul 21, 2016 at 07:54:48AM +0200, Roman Dobosz wrote:
> > On Wed, 20 Jul 2016 10:07:12 +0100
> > "Daniel P. Berrange" <berrange at redhat.com> wrote:
> >
> > Hey Daniel, thanks for the feedback.
> >
> > > > Thoughts?
> > >
> > > I'd suggest you'll increase your chances of success with nova design
> > > approval if you focus on implementing a really simple usage scheme for
> > > FPGA as the first step in Nova.
> >
> > This. Maybe I'm wrong, but for me the minimal use case for FPGA would
> > be ability to schedule VM which need certain accelerator from multiple
> > potential ones on available FPGA/fixed slot. How insane does it sound?
> >
> > Providing fixed, prepared earlier by DC administrator accelerator
> > resource, doesn't bring much value, beyond what we already have in
> > Nova, since PCI/SR-IOV passthrough might be used for accelerators,
> > which expose their functionality via VF.
>
> IIUC, there's plenty of FPGAs which are not SRIOV based, so there's
> still scope for Nova enhancement in this area.
>
> The fact that some FPGAs are SRIOV & some are not though, is is also
> why I'm suggesting that any work related to FPGA should be based around
> refactoring of the existing PCI device assignment model to form a more
> generic "Hardware device assignment" model.  If we end up having a
> completely distinct data model for FPGAs that is a failure. We need to
> have a generalized hardware assignment model that can be used for generic
> PCI devices, NICs, FPGAs, TPMs, GPUs, etc regardless of whether they
> are backed by SRIOV, or their own non-PCI virtual functions. Personally
> I'll reject any spec proposal that ignores existing PCI framework and
> introduces a separate model for FPGA.
>
> > > All the threads I've see go well off into the weeds about trying to
> > > solve everybody's niche/edge cases  perfectly and as a result get
> > > very complicated.
> >
> > The topic is complicated :)
>
> Which is why i'm advising to not try to solve the perfect case and instead
> focus on getting something simple & good enough for common case.
>
I think the simple use cases can be covered today for PCIe SR-IOV config
easily and some number of VFs are applied to regions of a pre-initialized
board.  I know of successful deployments that do the initialization with
ironic and use nova to allocate the PCIe SR-IOV access using existing
extension points. Once allocated the actual function bitstream gets pushed
in by the owning VM. The application owners manage concurrency. This level
of support could be made mainstream rather than custom extension as a first
step and then add support for alternatives to PCIe based connections.

That said there are many use cases in play today outside of openstack
unfortunately that manage the loading of the bitstream that implements a
specific function. The desire is to load those bitstreams and manage a life
cycle just like we manage a VM and image today. In effect the static region
of the FPGA has the role of a very simple hypervisor.

FPGA boards are getting denser and more common, and they are getting their
own peripherals like on board NICs, serial ports, storage etc.

I don't believe we need to expose complicated physical structure to
management, but a device with the ability to be virtualized and dynamically
programmed  and has connection to the other infrastructure in the
environment needs to be managed withe things it connects to.

I suggest the following :
First standardize how to describe and allocate a real or virtualized FPGA.
Specify the meta data and related filter rules.
Second, mirror the glance/nova process of image loading on hypervisor for
bitstream loading of a reProgramable Region.
Third keep the functions of the actual bit stream separate from the above
management just like we do with VM or container functional capabilities.

When the lifecycle of the PR is tied to a VM, just like ephemeral storage,
driving allocation from nova seems to make the most sense.

Am I way out of line?
> Regards,
> Daniel
> --
> |: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/
:|
> |: http://libvirt.org              -o-             http://virt-manager.org
:|
> |: http://autobuild.org       -o-         http://search.cpan.org/~danberr/
:|
> |: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc
:|
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160721/f9c9a9dd/attachment.html>