<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
worth also mentioning that MIG isn't currently supported in Openstack. We just finished a POC with some a100 and a40 cards and the vgpu setup wasn't too hard to do, but you definitely needed to read a combination of the NVIDIA docs and the openstack docs to
get a working setup. </div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
There is one open bug for mediated devices that you should be aware of though it looks like a fix is in the works:</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<a href="https://bugs.launchpad.net/nova/+bug/1900800" id="LPNoLPOWALinkPreview">https://bugs.launchpad.net/nova/+bug/1900800</a><br>
</div>
<div class="_Entity _EType_OWALinkPreview _EId_OWALinkPreview _EReadonly_1"></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
For straight PCI passthrough I found some of the tooling around it in openstack lacking compared to for the vgpu devices but that may have just been me missing something. Eg, I could only really see the PCI devices in the DB but couldn't find a way to see
them using the SDK / cli. </div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Sean Mooney <smooney@redhat.com><br>
<b>Sent:</b> 10 August 2022 11:23<br>
<b>To:</b> Alvaro Soto <alsotoes@gmail.com><br>
<b>Cc:</b> KK CHN <kkchn.in@gmail.com>; openstack-discuss <openstack-discuss@lists.openstack.org><br>
<b>Subject:</b> Re: Openstack Graphical processors virtualisation</font>
<div> </div>
</div>
<div>CAUTION: This email originates from outside THG<br>
<br>
our main vgpu expert is on PTO for a few weeks but ill try and respond inline.<br>
<br>
On Wed, Aug 10, 2022 at 8:07 AM Alvaro Soto <alsotoes@gmail.com> wrote:<br>
><br>
> Hey, just a little help with one question.<br>
><br>
> 1-<br>
> <a href="https://docs.openstack.org/nova/yoga/admin/virtual-gpu.html">
https://docs.openstack.org/nova/yoga/admin/virtual-gpu.html</a><br>
> <a href="https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/13/html/instances_and_images_guide/ch-virtual_gpu">
https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/13/html/instances_and_images_guide/ch-virtual_gpu</a><br>
yep thos are the docs for nova's vGPU support we also support pci<br>
passthough of a full gpu to a vm for usecause that need that.<br>
><br>
> On Wed, Aug 10, 2022 at 1:17 AM KK CHN <kkchn.in@gmail.com> wrote:<br>
>><br>
>> 1. Does Openstack support GPU virtualization ? We are running a cloud using ussuri with KVM hypervisors. Is creation of vGPUs possible?<br>
<br>
yes if you have an nvidia gpu and pay for there licnese server ectra<br>
you can config nova to expose those vgpus to the guests.<br>
if you have amd gpus and those support there mxgpu feature then that<br>
just exposes the vGPUs as stanard sriov VFs instead of vfio-mediated<br>
devices<br>
so you use pci passthough instead of the vGPU mdev feature in that<br>
case to consume them.<br>
>><br>
>> Does KVM support GPU virtualization or any limitations?<br>
yes however nvidia have not yet upstream support for vgpu/mdev live<br>
migration. redhat nvidia and others are currently activly working on<br>
upstreamoing that to the kernel<br>
qemu and libvirt but that is the main limiation. depending on your<br>
release nova also does not support move operations like cold migration<br>
however those have been<br>
added in more recent releases.<br>
>><br>
>> 2. What are the supported GPU types? (GRID GPU or GPUs attached to physical blades )<br>
both<br>
>><br>
>> 3. If it supports GPU attached to physical blades, will live migration of the VM be supported.Will OpenStack be able to identify the next host which has an attached GPU and perform live migration, in case of the Failure of One Blade with attached GPU with
resident VM.<br>
<br>
no live migration is not possibel with mdev or sriov vf attachment currently.<br>
cold migration and evacuate are supported in more recent releases of<br>
openstack in the case of maintance or hardware failure.<br>
>><br>
>> 4. What are the virtualization options if vGPU options are not supported by a particular GPU model?<br>
pci passthough of the fulll gpu or sriov if the card supprots that.<br>
>><br>
>><br>
>> Thanks in advance,<br>
>> Krish<br>
><br>
><br>
><br>
> --<br>
><br>
> Alvaro Soto<br>
><br>
> Note: My work hours may not be your work hours. Please do not feel the need to respond during a time that is not convenient for you.<br>
> ----------------------------------------------------------<br>
> Great people talk about ideas,<br>
> ordinary people talk about things,<br>
> small people talk... about other people.<br>
<br>
</div>
<div></div>
<p><span style="font-size:10.5pt;font-family:"Arial","sans-serif";color:black">Danny Webb</span>
<br>
<span style="font-size:8.5pt;font-family:"Arial","sans-serif";color:black">Principal OpenStack Engineer</span><br>
<span style="font-size:8.5pt;font-family:"Arial","sans-serif";color:#019EEB"><a href="http://www.thehutgroup.com/" target="_blank"><span style="color:#575a5d;
text-decoration:none;text-underline:none">The Hut Group</span></a></span>
<br>
<br>
<span style="font-size:8.5pt;font-family:"Arial","sans-serif";color:black">Tel: </span>
<br>
<span style="font-size:8.5pt;font-family:"Arial","sans-serif";color:black">Email:
<a href="mailto:Danny.Webb@thehutgroup.com">Danny.Webb@thehutgroup.com</a></span></p>
<p style="margin-bottom:12.0pt"><br>
<span style="font-size:7.5pt;font-family:"Arial","sans-serif";color:#666666">For the purposes of this email, the "company" means The Hut Group Limited, a company registered in England and Wales (company number 6539496) whose registered office is at Fifth Floor,
Voyager House, Chicago Avenue, Manchester Airport, M90 3DQ and/or any of its respective subsidiaries.</span>
<br>
<br>
<b><span style="font-size:7.5pt;font-family:"Arial","sans-serif";color:#666666">Confidentiality Notice</span></b>
<br>
<span style="font-size:7.5pt;font-family:"Arial","sans-serif";color:#666666">This e-mail is confidential and intended for the use of the named recipient only. If you are not the intended recipient please notify us by telephone immediately on +44(0)1606 811888
or return it to us by e-mail. Please then delete it from your system and note that any use, dissemination, forwarding, printing or copying is strictly prohibited. Any views or opinions are solely those of the author and do not necessarily represent those of
the company.</span> <br>
<br>
<b><span style="font-size:7.5pt;font-family:"Arial","sans-serif";color:#666666">Encryptions and Viruses</span></b>
<br>
<span style="font-size:7.5pt;font-family:"Arial","sans-serif";color:#666666">Please note that this e-mail and any attachments have not been encrypted. They may therefore be liable to be compromised. Please also note that it is your responsibility to scan this
e-mail and any attachments for viruses. We do not, to the extent permitted by law, accept any liability (whether in contract, negligence or otherwise) for any virus infection and/or external compromise of security and/or confidentiality in relation to transmissions
sent by e-mail.</span> <br>
<br>
<b><span style="font-size:7.5pt;font-family:"Arial","sans-serif";color:#666666">Monitoring</span></b>
<br>
<span style="font-size:7.5pt;font-family:"Arial","sans-serif";color:#666666">Activity and use of the company's systems is monitored to secure its effective use and operation and for other lawful business purposes. Communications using these systems will also
be monitored and may be recorded to secure effective use and operation and for other lawful business purposes.</span>
</p>
<span style="font-size:4pt;color:#FFFFFF">hgvyjuv</span>
<div></div>
</body>
</html>