Open Stack

Hi Andrew, hi John -

I've just started trying to get CUDA P2P working in our virtualized
HPC environment. I figure this must be something you solved already in
order to produce the aforementioned paper, but having read it a couple
of times I don't think it provides enough detail about the guest
config, hoping you can shed some light...

The issue I'm grappling with is that despite using a qemu-kvm machine
type (q35) with an emulated PCIe bus and seeing that indeed the P2P
capable GPUs (NVIDIA K80s) are attached to that bus, and nvidia-smi
sees them as sharing a PHB, the simpleP2P CUDA sample fails when
checking their ability to communicate with each other. Is there some
magic config I might be missing, did you need to make any PCI-ACS
changes?

Best regards,
Blair


On 16 March 2016 at 07:57, Blair Bethwaite <blair.bethwaite at gmail.com> wrote:
>
> Hi Andrew,
>
> On 16 March 2016 at 05:28, Andrew J Younge <ajyounge at indiana.edu> wrote:
> > point to a recent publication of ours at VEE15 titled "Supporting High
> > Performance Molecular Dynamics in Virtualized Clusters using IOMMU,
> > SR-IOV, and GPUDirect."  In the paper we show that using Nvidia GPUs
> ...
> > http://dl.acm.org/citation.cfm?id=2731194
>
> Oooh interesting - GPUDirect too. That's something I've been wanting
> to try out in our environment. Will take a look a your paper...
>
> --
> Cheers,
> ~Blairo


-- 
Cheers,
~Blairo