Hi Satish
I am not able to understand what is wrong with your environment, but I can describe my setting.
I have a compute node with 4 Tesla V100S. They have the same vendor-id (10de) and the same product id (13d6) [*] In nova.conf I defined this stuff in the [pci] section:
[pci] passthrough_whitelist = {"vendor_id":"10de"} alias={"name":"V100","product_id":"1df6","vendor_id":"10de","device_type":"type-PCI"}
I then created a flavor with this property:
pci_passthrough:alias='V100:1'
Using this flavor I can instantiate 4 VMs: each one can see a single V100
Hope this helps
Cheers, Massimo
[*] # lspci -nnk -d 10de: 60:00.0 3D controller [0302]: NVIDIA Corporation GV100GL [Tesla V100S PCIe 32GB] [10de:1df6] (rev a1) Subsystem: NVIDIA Corporation Device [10de:13d6] Kernel driver in use: vfio-pci Kernel modules: nouveau 61:00.0 3D controller [0302]: NVIDIA Corporation GV100GL [Tesla V100S PCIe 32GB] [10de:1df6] (rev a1) Subsystem: NVIDIA Corporation Device [10de:13d6] Kernel driver in use: vfio-pci Kernel modules: nouveau da:00.0 3D controller [0302]: NVIDIA Corporation GV100GL [Tesla V100S PCIe 32GB] [10de:1df6] (rev a1) Subsystem: NVIDIA Corporation Device [10de:13d6] Kernel driver in use: vfio-pci Kernel modules: nouveau db:00.0 3D controller [0302]: NVIDIA Corporation GV100GL [Tesla V100S PCIe 32GB] [10de:1df6] (rev a1) Subsystem: NVIDIA Corporation Device [10de:13d6] Kernel driver in use: vfio-pci Kernel modules: nouveau [root@cld-np-gpu-01 ~]#
On Wed, Jan 19, 2022 at 10:28 PM Satish Patel satish.txt@gmail.com wrote:
Hi Massimo,
Ignore my last email, my requirement is to have a single VM with a single GPU ("tesla-v100:1") but I would like to create a second VM on the same compute node which uses the second GPU but I am getting the following error when I create a second VM and vm error out. looks like it's not allowing me to create a second vm and bind to a second GPU card.
error : virDomainDefDuplicateHostdevInfoValidate:1082 : XML error: Hostdev already exists in the domain configuration
On Wed, Jan 19, 2022 at 3:10 PM Satish Patel satish.txt@gmail.com wrote:
should i need to create a flavor to target both GPU. is it possible to have single flavor cover both GPU because end users don't understand which flavor to use.
On Wed, Jan 19, 2022 at 1:54 AM Massimo Sgaravatto massimo.sgaravatto@gmail.com wrote:
If I am not wrong those are 2 GPUs
"tesla-v100:1" means 1 GPU
So e.g. a flavor with "pci_passthrough:alias": "tesla-v100:2"} will be
used to create an instance with 2 GPUs
Cheers, Massimo
On Tue, Jan 18, 2022 at 11:35 PM Satish Patel satish.txt@gmail.com
wrote:
Thank you for the information. I have a quick question.
[root@gpu01 ~]# lspci | grep -i nv 5e:00.0 3D controller: NVIDIA Corporation GV100GL [Tesla V100S PCIe 32GB] (rev a1) d8:00.0 3D controller: NVIDIA Corporation GV100GL [Tesla V100S PCIe 32GB] (rev a1)
In the above output showing two cards does that mean they are physical two or just BUS representation.
Also i have the following entry in openstack flavor, does :1 means first GPU card?
{"gpu-node": "true", "pci_passthrough:alias": "tesla-v100:1"}
On Tue, Jan 18, 2022 at 5:55 AM António Paulo antonio.paulo@cern.ch
wrote:
Hey Satish, Gustavo,
Just to clarify a bit on point 3, you will have to buy a vGPU
license
per card and this gives you access to all the downloads you need
through
NVIDIA's web dashboard -- both the host and guest drivers as well
as the
license server setup files.
Cheers, António
On 18/01/22 02:46, Satish Patel wrote:
Thank you so much! This is what I was looking for. It is very odd
that
we buy a pricey card but then we have to buy a license to make
those
features available.
On Mon, Jan 17, 2022 at 2:07 PM Gustavo Faganello Santos gustavofaganello.santos@windriver.com wrote: > > Hello, Satish. > > I've been working with vGPU lately and I believe I can answer
your
> questions: > > 1. As you pointed out in question #2, the pci-passthrough will
allocate
> the entire physical GPU to one single guest VM, while vGPU
allows you to
> spawn from 1 to several VMs using the same physical GPU,
depending on
> the vGPU type you choose (check NVIDIA docs to see which vGPU
types the
> Tesla V100 supports and their properties); > 2. Correct; > 3. To use vGPU, you need vGPU drivers installed on the platform
where
> your deployment of OpenStack is running AND in the VMs, so there
are two
> drivers to be installed in order to use the feature. I believe
both of
> them have to be purchased from NVIDIA in order to be used, and
you would
> also have to deploy an NVIDIA licensing server in order to
validate the
> licenses of the drivers running in the VMs. > 4. You can see what the instructions are for each of these
scenarios in
> [1] and [2]. > > There is also extensive documentation on vGPU at NVIDIA's
website [3].
> > [1]
https://docs.openstack.org/nova/wallaby/admin/virtual-gpu.html
> [2]
https://docs.openstack.org/nova/wallaby/admin/pci-passthrough.html
> [3] https://docs.nvidia.com/grid/13.0/index.html > > Regards, > Gustavo. > > On 17/01/2022 14:41, Satish Patel wrote: >> [Please note: This e-mail is from an EXTERNAL e-mail address] >> >> Folk, >> >> We have Tesla V100 32G GPU and I’m trying to configure with
openstack wallaby. This is first time dealing with GPU so I have couple of question.
>> >> 1. What is the difference between passthrough vs vGPU? I did
google but not very clear yet.
>> 2. If I configure it passthrough then does it only work with
single VM ? ( I meant whole GPU will get allocate to single VM correct?
>> 3. Also some document saying Tesla v100 support vGPU but some
folks saying you need license. I have no idea where to get that license. What is the deal here?
>> 3. What are the config difference between configure this card
with passthrough vs vGPU?
>> >> >> Currently I configure it with passthrough based one one article
and I am able to spun up with and I can see nvidia card exposed to vm. (I used iommu and vfio based driver) so if this card support vGPU then do I need iommu and vfio or some other driver to make it virtualize ?
>> >> Sent from my iPhone >>