[openstack-ansible]: GPU passthrough
Hi Experts Release: Wallaby Distribution: Openstack-ansible I am working on enabling GPU computes through PCIpassthrough: I managed to test the GPU PCI passthrough with the direct configuration below. *GPU compute:* /etc/nova/nova.conf [pci] device_spec = { "vendor_id": "10de", "product_id": "25b6" } passthrough_whitelist = { "vendor_id": "10de", "product_id": "25b6" } alias = { "vendor_id": "10de", "product_id": "25b6", "device_type": "type-PCI", "name": "nvidia" } *Nova API container:* /etc/nova/nova.conf enabled_filters = AvailabilityZoneFilter,ComputeFilter,AggregateNumInstancesFilter,AggregateIoOpsFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,NUMATopologyFilter, *PciPassthroughFilter* [pci] alias = { "vendor_id": "10de", "product_id": "25b6", "device_type": "type-PF", "name": "nvidia" } Now I need to put the configuration through openstack-ansible. So I need support for the same. *compute:* I intend to use hostvar for the compute configuration: /etc/openstack_deploy/host_vars/compute-<>.yml Can someone please confirm the parameters I need to set in this file and how to implement it? device_spec = ? passthrough_whitelist = ? nova_pci_alias: - '{ "name": "nvidia", "product_id": "25b6", "vendor_id": "10de" }' =====> how can I add *device_type field* here as per the mail.yaml it only have name,product_id and vendor_id field? Run openstack-ansible os-nova-install.yml --tags nova-config --limit <compute name> *For nova_api configuration,* I intend to use below: Below parameter in the user_variable file nova_scheduler_default_filters: - AvailabilityZoneFilter - ComputeFilter - AggregateInstanceExtraSpecsFilter - AggregateNumInstancesFilter - AggregateIoOpsFilter - ComputeCapabilitiesFilter - ImagePropertiesFilter - ServerGroupAntiAffinityFilter - ServerGroupAffinityFilter - NUMATopologyFilter - PciPassthroughFilter nova_pci_alias: - '{ "name": "nvidia", "product_id": "25b6", "vendor_id": "10de" }' =====> how can I add *device_type field here* as per the mail.yaml it only have name,product_id and vendor_id field? Execute: openstack-ansible os-nova-install.yml --tags nova-config --limit <nova api containers> Regards
Hey, We actually had some refactoring of this code in later releases but it's not there for Wallaby yet. So - nova_pci_alias is a regular list, you can define any content inside of it and it will be populated in the template. As we do not parse the content - just loop over the list: https://opendev.org/openstack/openstack-ansible-os_nova/src/branch/unmaintai... There is also a variable for nova_pci_passthrough_whitelist - which will also paste content pretty much "as is". However, there's no `device_spec` in Wallaby - we got it implemented only in Zed: https://review.opendev.org/q/Id2da29464359b4845c7d05e3bec53759341f4bad So one thing I usually suggest doing in such cases - using overrides. There is a variable nova_nova_conf_overrides which can be used to add arbitrary content to nova.conf (we have such variables for pretty much every config file). You can define it as: nova_nova_conf_overrides: pci: device_spec: '{ "vendor_id": "10de", "product_id": "25b6" }' Eventually, you can also just ignore nova_pci_passthrough_whitelist and nova_pci_alias, and just use overrides for them as well, ie: nova_nova_conf_overrides: pci: device_spec: '{ "vendor_id": "10de", "product_id": "25b6" }' passthrough_whitelist: '{ "vendor_id": "10de", "product_id": "25b6" }' alias: '{ "vendor_id": "10de", "product_id": "25b6", "device_type": "type-PCI", "name": "nvidia" }' Just make sure, that you don't have `nova_nova_conf_overrides` defined elsewhere (like user_variables), as then the variable with higher precedence will "win". Though, you can do some "combine" from different group_vars/host_vars or just place full override content to host_vars... вт, 21 янв. 2025 г. в 22:47, Rambo Rambo <ram.ramb2412@gmail.com>:
Hi Experts
Release: Wallaby Distribution: Openstack-ansible
I am working on enabling GPU computes through PCIpassthrough:
I managed to test the GPU PCI passthrough with the direct configuration below.
GPU compute: /etc/nova/nova.conf
[pci] device_spec = { "vendor_id": "10de", "product_id": "25b6" } passthrough_whitelist = { "vendor_id": "10de", "product_id": "25b6" } alias = { "vendor_id": "10de", "product_id": "25b6", "device_type": "type-PCI", "name": "nvidia" }
Nova API container: /etc/nova/nova.conf
enabled_filters = AvailabilityZoneFilter,ComputeFilter,AggregateNumInstancesFilter,AggregateIoOpsFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,NUMATopologyFilter,PciPassthroughFilter
[pci] alias = { "vendor_id": "10de", "product_id": "25b6", "device_type": "type-PF", "name": "nvidia" }
Now I need to put the configuration through openstack-ansible. So I need support for the same.
compute:
I intend to use hostvar for the compute configuration:
/etc/openstack_deploy/host_vars/compute-<>.yml
Can someone please confirm the parameters I need to set in this file and how to implement it?
device_spec = ? passthrough_whitelist = ? nova_pci_alias: - '{ "name": "nvidia", "product_id": "25b6", "vendor_id": "10de" }' =====> how can I add device_type field here as per the mail.yaml it only have name,product_id and vendor_id field?
Run openstack-ansible os-nova-install.yml --tags nova-config --limit <compute name>
For nova_api configuration, I intend to use below:
Below parameter in the user_variable file
nova_scheduler_default_filters: - AvailabilityZoneFilter - ComputeFilter - AggregateInstanceExtraSpecsFilter - AggregateNumInstancesFilter - AggregateIoOpsFilter - ComputeCapabilitiesFilter - ImagePropertiesFilter - ServerGroupAntiAffinityFilter - ServerGroupAffinityFilter - NUMATopologyFilter - PciPassthroughFilter
nova_pci_alias: - '{ "name": "nvidia", "product_id": "25b6", "vendor_id": "10de" }' =====> how can I add device_type field here as per the mail.yaml it only have name,product_id and vendor_id field?
Execute: openstack-ansible os-nova-install.yml --tags nova-config --limit <nova api containers>
Regards
Hi Dmitriy Thanks alot for the response managed to implement the GPU configuration using openstack-ansible. Regards On Wed, Jan 22, 2025 at 10:46 AM Dmitriy Rabotyagov <noonedeadpunk@gmail.com> wrote:
Hey,
We actually had some refactoring of this code in later releases but it's not there for Wallaby yet.
So - nova_pci_alias is a regular list, you can define any content inside of it and it will be populated in the template. As we do not parse the content - just loop over the list:
https://opendev.org/openstack/openstack-ansible-os_nova/src/branch/unmaintai...
There is also a variable for nova_pci_passthrough_whitelist - which will also paste content pretty much "as is". However, there's no `device_spec` in Wallaby - we got it implemented only in Zed: https://review.opendev.org/q/Id2da29464359b4845c7d05e3bec53759341f4bad
So one thing I usually suggest doing in such cases - using overrides. There is a variable nova_nova_conf_overrides which can be used to add arbitrary content to nova.conf (we have such variables for pretty much every config file). You can define it as:
nova_nova_conf_overrides: pci: device_spec: '{ "vendor_id": "10de", "product_id": "25b6" }'
Eventually, you can also just ignore nova_pci_passthrough_whitelist and nova_pci_alias, and just use overrides for them as well, ie:
nova_nova_conf_overrides: pci: device_spec: '{ "vendor_id": "10de", "product_id": "25b6" }' passthrough_whitelist: '{ "vendor_id": "10de", "product_id": "25b6" }' alias: '{ "vendor_id": "10de", "product_id": "25b6", "device_type": "type-PCI", "name": "nvidia" }'
Just make sure, that you don't have `nova_nova_conf_overrides` defined elsewhere (like user_variables), as then the variable with higher precedence will "win". Though, you can do some "combine" from different group_vars/host_vars or just place full override content to host_vars...
вт, 21 янв. 2025 г. в 22:47, Rambo Rambo <ram.ramb2412@gmail.com>:
Hi Experts
Release: Wallaby Distribution: Openstack-ansible
I am working on enabling GPU computes through PCIpassthrough:
I managed to test the GPU PCI passthrough with the direct configuration
below.
GPU compute: /etc/nova/nova.conf
[pci] device_spec = { "vendor_id": "10de", "product_id": "25b6" } passthrough_whitelist = { "vendor_id": "10de", "product_id": "25b6" } alias = { "vendor_id": "10de", "product_id": "25b6", "device_type":
"type-PCI", "name": "nvidia" }
Nova API container: /etc/nova/nova.conf
enabled_filters =
AvailabilityZoneFilter,ComputeFilter,AggregateNumInstancesFilter,AggregateIoOpsFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,NUMATopologyFilter,PciPassthroughFilter
[pci] alias = { "vendor_id": "10de", "product_id": "25b6", "device_type":
"type-PF", "name": "nvidia" }
Now I need to put the configuration through openstack-ansible. So I need
support for the same.
compute:
I intend to use hostvar for the compute configuration:
/etc/openstack_deploy/host_vars/compute-<>.yml
Can someone please confirm the parameters I need to set in this file and
how to implement it?
device_spec = ? passthrough_whitelist = ? nova_pci_alias: - '{ "name": "nvidia", "product_id": "25b6", "vendor_id": "10de" }'
=====> how can I add device_type field here as per the mail.yaml it only have name,product_id and vendor_id field?
Run openstack-ansible os-nova-install.yml --tags nova-config --limit
<compute name>
For nova_api configuration, I intend to use below:
Below parameter in the user_variable file
nova_scheduler_default_filters: - AvailabilityZoneFilter - ComputeFilter - AggregateInstanceExtraSpecsFilter - AggregateNumInstancesFilter - AggregateIoOpsFilter - ComputeCapabilitiesFilter - ImagePropertiesFilter - ServerGroupAntiAffinityFilter - ServerGroupAffinityFilter - NUMATopologyFilter - PciPassthroughFilter
nova_pci_alias: - '{ "name": "nvidia", "product_id": "25b6", "vendor_id": "10de" }'
=====> how can I add device_type field here as per the mail.yaml it only have name,product_id and vendor_id field?
Execute: openstack-ansible os-nova-install.yml --tags nova-config --limit <nova
api containers>
Regards
participants (2)
-
Dmitriy Rabotyagov
-
Rambo Rambo