<p dir="ltr">On Jul 21, 2016 5:12 AM, "Daniel P. Berrange" <<a href="mailto:berrange@redhat.com">berrange@redhat.com</a>> wrote:<br>
><br>
> On Thu, Jul 21, 2016 at 07:54:48AM +0200, Roman Dobosz wrote:<br>
> > On Wed, 20 Jul 2016 10:07:12 +0100<br>
> > "Daniel P. Berrange" <<a href="mailto:berrange@redhat.com">berrange@redhat.com</a>> wrote:<br>
> ><br>
> > Hey Daniel, thanks for the feedback.<br>
> ><br>
> > > > Thoughts?<br>
> > ><br>
> > > I'd suggest you'll increase your chances of success with nova design<br>
> > > approval if you focus on implementing a really simple usage scheme for<br>
> > > FPGA as the first step in Nova.<br>
> ><br>
> > This. Maybe I'm wrong, but for me the minimal use case for FPGA would<br>
> > be ability to schedule VM which need certain accelerator from multiple<br>
> > potential ones on available FPGA/fixed slot. How insane does it sound?<br>
> ><br>
> > Providing fixed, prepared earlier by DC administrator accelerator<br>
> > resource, doesn't bring much value, beyond what we already have in<br>
> > Nova, since PCI/SR-IOV passthrough might be used for accelerators,<br>
> > which expose their functionality via VF.<br>
><br>
> IIUC, there's plenty of FPGAs which are not SRIOV based, so there's<br>
> still scope for Nova enhancement in this area.<br>
><br>
> The fact that some FPGAs are SRIOV & some are not though, is is also<br>
> why I'm suggesting that any work related to FPGA should be based around<br>
> refactoring of the existing PCI device assignment model to form a more<br>
> generic "Hardware device assignment" model. If we end up having a<br>
> completely distinct data model for FPGAs that is a failure. We need to<br>
> have a generalized hardware assignment model that can be used for generic<br>
> PCI devices, NICs, FPGAs, TPMs, GPUs, etc regardless of whether they<br>
> are backed by SRIOV, or their own non-PCI virtual functions. Personally<br>
> I'll reject any spec proposal that ignores existing PCI framework and<br>
> introduces a separate model for FPGA.<br>
><br>
> > > All the threads I've see go well off into the weeds about trying to<br>
> > > solve everybody's niche/edge cases perfectly and as a result get<br>
> > > very complicated.<br>
> ><br>
> > The topic is complicated :)<br>
><br>
> Which is why i'm advising to not try to solve the perfect case and instead<br>
> focus on getting something simple & good enough for common case.<br>
><br>
I think the simple use cases can be covered today for PCIe SR-IOV config easily and some number of VFs are applied to regions of a pre-initialized board. I know of successful deployments that do the initialization with ironic and use nova to allocate the PCIe SR-IOV access using existing extension points. Once allocated the actual function bitstream gets pushed in by the owning VM. The application owners manage concurrency. This level of support could be made mainstream rather than custom extension as a first step and then add support for alternatives to PCIe based connections. </p>
<p dir="ltr">That said there are many use cases in play today outside of openstack unfortunately that manage the loading of the bitstream that implements a specific function. The desire is to load those bitstreams and manage a life cycle just like we manage a VM and image today. In effect the static region of the FPGA has the role of a very simple hypervisor. </p>
<p dir="ltr">FPGA boards are getting denser and more common, and they are getting their own peripherals like on board NICs, serial ports, storage etc.</p>
<p dir="ltr">I don't believe we need to expose complicated physical structure to management, but a device with the ability to be virtualized and dynamically programmed and has connection to the other infrastructure in the environment needs to be managed withe things it connects to. <br>
<br>
I suggest the following :<br>
First standardize how to describe and allocate a real or virtualized FPGA. Specify the meta data and related filter rules. <br>
Second, mirror the glance/nova process of image loading on hypervisor for bitstream loading of a reProgramable Region. <br>
Third keep the functions of the actual bit stream separate from the above management just like we do with VM or container functional capabilities. </p>
<p dir="ltr">When the lifecycle of the PR is tied to a VM, just like ephemeral storage, driving allocation from nova seems to make the most sense. </p>
<p dir="ltr">Am I way out of line?<br>
> Regards,<br>
> Daniel<br>
> --<br>
> |: <a href="http://berrange.com">http://berrange.com</a> -o- <a href="http://www.flickr.com/photos/dberrange/">http://www.flickr.com/photos/dberrange/</a> :|<br>
> |: <a href="http://libvirt.org">http://libvirt.org</a> -o- <a href="http://virt-manager.org">http://virt-manager.org</a> :|<br>
> |: <a href="http://autobuild.org">http://autobuild.org</a> -o- <a href="http://search.cpan.org/~danberr/">http://search.cpan.org/~danberr/</a> :|<br>
> |: <a href="http://entangle-photo.org">http://entangle-photo.org</a> -o- <a href="http://live.gnome.org/gtk-vnc">http://live.gnome.org/gtk-vnc</a> :|<br>
><br>
> __________________________________________________________________________<br>
> OpenStack Development Mailing List (not for usage questions)<br>
> Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</p>