Ended up with the very same conclusions than Dimitry regarding the use of Nvidia Vgrid for the VGPU use case with Nova, it works pretty well but:
- respecting the licensing model as operationnal constraints, note that guests need to reach a license server in order to get a token (could be via the Nvidia SaaS service or on-prem)
- drivers for both guest and hypervisor are not easy to implement and maintain on large scale. A year ago, hypervisors drivers were not packaged to Debian/Ubuntu, but builded though a bash script, thus requiering additional automatisation work and careful attention regarding kernel update/reboot of Nova hypervisors.
Cheers
On Fri, Jan 13, 2023 at 4:21 PM Dmitriy Rabotyagov <noonedeadpunk@gmail.com> wrote:
>
> You are saying that, like Nvidia GRID drivers are open-sourced while
> in fact they're super far from being that. In order to download
> drivers not only for hypervisors, but also for guest VMs you need to
> have an account in their Enterprise Portal. It took me roughly 6 weeks
> of discussions with hardware vendors and Nvidia support to get a
> proper account there. And that happened only after applying for their
> Partner Network (NPN).
> That still doesn't solve the issue of how to provide drivers to
> guests, except pre-build a series of images with these drivers
> pre-installed (we ended up with making a DIB element for that [1]).
> Not saying about the need to distribute license tokens for guests and
> the whole mess with compatibility between hypervisor and guest drivers
> (as guest driver can't be newer then host one, and HVs can't be too
> new either).
>
> It's not that I'm protecting AMD, but just saying that Nvidia is not
> that straightforward either, and at least on paper AMD vGPUs look
> easier both for operators and end-users.
>
> [1] https://github.com/citynetwork/dib-elements/tree/main/nvgrid
>
> >
> > As for AMD cards, AMD stated that some of their MI series card supports SR-IOV for vGPUs. However, those drivers are never open source or provided closed source to public, only large cloud providers are able to get them. So I don't really recommend getting AMD cards for vGPU unless you are able to get support from them.
> >
>