<html><body><p>Hi all,<br><br>We agree this is an ingesting direction we can do exploration on OpenStack and actually we already have such FPGA/GPU as a resource in our public research cloud called SuperVessel (<a href="https://ptopenlab.com/cloudlabconsole/#/">https://ptopenlab.com</a>). With modification of OpenStack from community, nearly all features mentioned by <tt>Roman Dobosz</tt> are supported, such as FPGA boards detection, scheduling, images management and user API layer. Here are some materials for you to reference [1][2].<br><br>Unlike CPU and GPU, right now FPGA accelerator is developed without common protocol above PCIe protocol. Along with huge kinds of FPGA chips, FPGA boards, it is hard to be managed. Passing through the FPGA as a general PCIe device to the application stack is a straight way to expose the FPGA acceleration ability in cloud while it is far way to make it easy used. It dose not help the accelerator detection, reload, reconfiguration, deployment in cloud.<br><br>Good news is that, the FPGA community is actively promoting high level programming language, such as OpenCL. They hide nearly all hardware detail for the developers so it make the "accelerator" common in different hardware. It is reasonable to see the FPGA become a popular acceleration hardware. It also reasonable for OpenStack community begin the discussion to support the FPGA resource management.<br><br><br>[1] <a href="http://openpowerfoundation.org/blogs/fpga-acceleration-in-a-power8-cloud/">http://openpowerfoundation.org/blogs/fpga-acceleration-in-a-power8-cloud/</a><br>[2] <a href="http://dl.acm.org/citation.cfm?id=2597929">http://dl.acm.org/citation.cfm?id=2597929</a><br><br><br><br>Best regards,<br>CHEN Fei<br><br><br><br><img width="16" height="16" src="cid:1__=8FBBF51DDF97553F8f9e8a93df938690918c8FB@" border="0" alt="Inactive hide details for Qiming Teng ---2016/04/06 12:25:17---Emm... finally this is brought up. We from IBM have already done"><font color="#424282">Qiming Teng ---2016/04/06 12:25:17---Emm... finally this is brought up. We from IBM have already done some work on FPGA/GPU resource mana</font><br><br><font size="2" color="#5F5F5F">From: </font><font size="2">Qiming Teng <tengqim@linux.vnet.ibm.com></font><br><font size="2" color="#5F5F5F">To: </font><font size="2">"OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org></font><br><font size="2" color="#5F5F5F">Cc: </font><font size="2">Fei K Chen/China/IBM@IBMCN, Yong Hua Lin/China/IBM@IBMCN</font><br><font size="2" color="#5F5F5F">Date: </font><font size="2">2016/04/06 12:25</font><br><font size="2" color="#5F5F5F">Subject: </font><font size="2">Re: [openstack-dev] [Nova] FPGA as a resource</font><br><hr width="100%" size="2" align="left" noshade style="color:#8091A5; "><br><br><br><tt>Emm... finally this is brought up. We from IBM have already done some<br>work on FPGA/GPU resource management [1]. Let me bring the SMEs into<br>this discussion and see if we together can work out a concrete roadmap<br>to land this upstream.<br><br>Fei and Yonghua, this is indeed very interesting a topic for us.<br><br><br>[1] SuperVessel Cloud: </tt><tt><a href="https://ptopenlab.com/">https://ptopenlab.com/</a></tt><tt><br><br>Regards,<br> Qiming<br><br>On Tue, Apr 05, 2016 at 02:27:30PM +0200, Roman Dobosz wrote:<br>> Hey all,<br>> <br>> On yesterday's scheduler meeting I was raised the idea of bringing up <br>> the FPGA to the OpenStack as the resource, which than might be exposed <br>> to the VMs.<br>> <br>> The use cases for motivations, why one want do this, are pretty broad -<br>> having such chip ready on the computes might be beneficial either for<br>> consumers of the technology and data center administrators. The<br>> utilization of the hardware is very broad - the only limitations are<br>> human imagination and hardware capability - since it might be used for<br>> accelerating execution of algorithms from compression and cryptography,<br>> through pattern recognition, transcoding to voice/video analysis and<br>> processing and all the others in between. Using FPGA to perform data<br>> processing may significantly reduce CPU utilization, the time and power<br>> consumption, which is a benefit on its own.<br>> <br>> On OpenStack side, unlike utilizing the CPU or memory, for actually <br>> using specific algorithm with FPGAs, it has to be programmed first. So <br>> in simplified scenario, it might go like this:<br>> <br>> * User selects VM with image which supports acceleration,<br>> * Scheduler selects the appropriate compute host with FPGA available,<br>> * Compute gets request, program IP into FPGA and then boot up the <br>> VM with accelerator attached.<br>> * If VM is removed, it may optionally erase the FPGA.<br>> <br>> As you can see, it seems not complicated at this point, however it <br>> become more complex due to following things we also have to take into <br>> consideration:<br>> <br>> * recent FPGA are divided into regions (or slots) and every of them <br>> can be programmed separately<br>> * slots may or may not fit the same bitstream (the program which FPGA<br>> is fed, the IP)<br>> * There is several products around (Altera, Xilinx, others), which make <br>> bitstream incompatible. Even between the products of the same company<br>> * libraries which abstract the hardware layer like AAL[1] and their <br>> versions<br>> * for some products, there is a need for tracking memory usage, which <br>> is located on PCI boards<br>> * some of the FPGAs can be exposed using SR-IOV, while some other not, <br>> which implies multiple usage abilities<br>> <br>> In other words, it may be necessary to incorporate another actions:<br>> <br>> * properly discover FPGA and its capabilities<br>> * schedule right bitstream with corresponding matching unoccupied FPGA<br>> device/slot<br>> * actual program FPGA<br>> * provide libraries into VM, which are necessary for interacting between<br>> user program and the exposed FPGA (or AAL) (this may be optional, <br>> since user can upload complete image with everything in place)<br>> * bitstream images have to be keep in some kind of service (Glance?) <br>> with some kind of way for identifying which image match what FPGA<br>> <br>> All of that makes modelling resource extremely complicated, contrary to <br>> CPU resource for example. I'd like to discuss how the goal of having <br>> reprogrammable accelerators in OpenStack can be achieved. Ideally I'd <br>> like to fit it into Jay and Chris work on resource-*.<br>> <br>> Looking forward for any comments :)<br>> <br>> [1] </tt><tt><a href="http://www.intel.com/content/dam/doc/white-paper/quickassist-technology-aal-white-paper.pdf">http://www.intel.com/content/dam/doc/white-paper/quickassist-technology-aal-white-paper.pdf</a></tt><tt><br>> <br>> -- <br>> Cheers,<br>> Roman Dobosz<br>> <br>> __________________________________________________________________________<br>> OpenStack Development Mailing List (not for usage questions)<br>> Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe<br>> </tt><tt><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></tt><tt><br>> <br></tt><br><BR>
</body></html>