Open Stack

On Tue, 2019-05-07 at 13:47 -0400, Artom Lifshitz wrote:
> Hey all,
> 
> Following up on the CI session during the PTG [1], I wanted to get the
> ball rolling on getting GPU hardware into the gate somehow. Initially
> the plan was to do it through OpenLab and by convincing NVIDIA to
> donate the cards, but after a conversation with Sean McGinnis it
> appears Infra have access to machines with GPUs.
> 
> From Nova's POV, the requirements are:
> * The machines with GPUs should probably be Ironic baremetal nodes and
> not VMs [*].
> * The GPUs need to support virtualization. It's hard to get a
> comprehensive list of GPUs that do, but Nova's own docs [2] mention
> two: Intel cards with GVT [3] and NVIDIA GRID [4].
Intel cards is a misnomer 
GVT is currently only supported by the integrated gpu on intel cpus which
was removed form xeon cpus when GVT support was added. in the future with
the descrete gpus from intel slated for release sometime in 2020 we should
hopefully have intel cards that actully support GVT assuming that is on there
gpu product roadmap but i can see how it would not be given they developed the
tech for there integrated gpu.

it would also be intersting to test amd gpus using there sriov approach but
i think NVIDA tesla gpus would be the shortest path forword.
> 
> So I think at this point the question is whether Infra can support
> those reqs. If yes, we can start concrete steps towards getting those
> machines used by a CI job. If not, we'll fall back to OpenLab and try
> to get them hardware.
> 
> [*] Could we do double-passthrough? Could the card be passed through
> to the L1 guest via the PCI passthrough mechanism, and then into the
> L2 guest via the mdev mechanism?

i have a theory about how this "migth" be possible but openstack is missing
the features requried to pull it off. i may test it locally with libvirt but
the only why i think this could work would be to do a full passthough of the PF
to an l1 guest using the q35 chipset with a viommu(not supported in nova) with hypervior
hiding enabled and then run the grid driver in the l1 guest to expose a mdev to the l2 guest.

ironic would be much simpler
> 
> [1] https://etherpad.openstack.org/p/nova-ptg-train-ci
> [2] https://docs.openstack.org/nova/rocky/admin/virtual-gpu.html
> [3] https://01.org/igvt-g
> [4] https://docs.nvidia.com/grid/5.0/pdf/grid-vgpu-user-guide.pdf
>