On 17/06/2025 09:30, Arnaud Morin wrote:
Hello nova team,
Quick question regarding support for multiple types (see [1]) with the same address: {vendor_id: "10de", product_id: "233b", "address": "00000000:03:00.0", "resource_class": "CUSTOM_H200_A"} {vendor_id: "10de", product_id: "233b", "address": "00000000:04:00.0", "resource_class": "CUSTOM_H200_A"} {vendor_id: "10de", product_id: "233b", "address": "00000000:44:00.0", "resource_class": "CUSTOM_H200_A"} {vendor_id: "10de", product_id: "233b", "address": "00000000:45:00.0", "resource_class": "CUSTOM_H200_A"} {vendor_id: "10de", product_id: "233b", "address": "00000000:83:00.0", "resource_class": "CUSTOM_H200_B"} {vendor_id: "10de", product_id: "233b", "address": "00000000:84:00.0", "resource_class": "CUSTOM_H200_B"} {vendor_id: "10de", product_id: "233b", "address": "00000000:C3:00.0", "resource_class": "CUSTOM_H200_B"} {vendor_id: "10de", product_id: "233b", "address": "00000000:C4:00.0", "resource_class": "CUSTOM_H200_B"}
This works fine, I was able to define multiple aliases: {name: "h200a", device_type:"type-PF", resource_class: "CUSTOM_H200_A"} {name: "h200b", device_type:"type-PF", resource_class: "CUSTOM_H200_B"}
I did that to create two blocks of 4 resources (A or B).
But now, I need to create a flavor to boot instances with these devices. I want to have only one flavor that can use either A or B: For now I created a flavor with: pci_passthrough:alias='h200a:4'
And was forced to create a second flavor with h200b:4.
Is there any way to achieve a single flavor with both: Something like this? pci_passthrough:alias='h200a:4|h200b:4'
I can't figure that out for now, is it possible?
no its not and for reasons is also not easy to implement that in the future because of how placement works. we would need a new OneOF type query in the allocation candidates api. the feature you really want is https://specs.openstack.org/openstack/nova-specs/specs/2024.1/approved/pci-p... it was proposed but never implemented. with that spec you can carve ups pci device into groups in the pci devspec and then have a a singel resouce class to select any one of the groups. so in your case you would express ``` {vendor_id: "10de", product_id: "233b", "address": "00000000:03:00.0", "resource_class": "CUSTOM_H200_A"} {vendor_id: "10de", product_id: "233b", "address": "00000000:04:00.0", "resource_class": "CUSTOM_H200_A"} {vendor_id: "10de", product_id: "233b", "address": "00000000:44:00.0", "resource_class": "CUSTOM_H200_A"} {vendor_id: "10de", product_id: "233b", "address": "00000000:45:00.0", "resource_class": "CUSTOM_H200_A"} {vendor_id: "10de", product_id: "233b", "address": "00000000:83:00.0", "resource_class": "CUSTOM_H200_B"} {vendor_id: "10de", product_id: "233b", "address": "00000000:84:00.0", "resource_class": "CUSTOM_H200_B"} {vendor_id: "10de", product_id: "233b", "address": "00000000:C3:00.0", "resource_class": "CUSTOM_H200_B"} {vendor_id: "10de", product_id: "233b", "address": "00000000:C4:00.0", "resource_class": "CUSTOM_H200_B"} ``` as ``` {vendor_id: "10de", product_id: "233b", "address": "00000000:03:00.0", "group_name": "A", "group_type": "H200"} {vendor_id: "10de", product_id: "233b", "address": "00000000:04:00.0", "group_name": "A", "group_type": "H200"} {vendor_id: "10de", product_id: "233b", "address": "00000000:44:00.0", "group_name": "A", "group_type": "H200"} {vendor_id: "10de", product_id: "233b", "address": "00000000:45:00.0", "group_name": "A", "group_type": "H200"} {vendor_id: "10de", product_id: "233b", "address": "00000000:83:00.0", "group_name": "B", "group_type": "H200"} {vendor_id: "10de", product_id: "233b", "address": "00000000:84:00.0", "group_name": "B", "group_type": "H200"} {vendor_id: "10de", product_id: "233b", "address": "00000000:C3:00.0", "group_name": "B", "group_type": "H200"} {vendor_id: "10de", product_id: "233b", "address": "00000000:C4:00.0", "group_name": "B", "group_type": "H200"} alias = {"name":"h200", resource_class:"CUSTOM_PCI_GROUP_H200"} ``` and in the flavor you would request pci_passthrough:alias='h200:1' pci groups allocate a full group to the request. its a very useful propal that we have discuss on an over for thet better part of a decade and finally got as far as writign it down for 2024.1 but then the implementation never got started. i know some redhat customer are also asked about this type of grouping functionality so at least internally this comes up form time to time. i strongly suspect this will eventually get implemtned but there have been higher priorites for the nova team like eventlet removal or gpu live migrations. if some one propsoes it again i know at least john garbutt was keen to see this added for some hpc usecases and i suspect with the current AI boom passing blocks of H200 gpus to a workload is becomming more common not less.
[1] https://docs.openstack.org/nova/latest/admin/pci-passthrough.html#support-fo...