Open Stack

From: Chinmaya Dwibedy [mailto:ckdwibedy at gmail.com]
Sent: Wednesday, June 08, 2016 8:39 AM
To: Moshe Levi <moshele at mellanox.com>
Cc: openstack at lists.openstack.org
Subject: Re: [Openstack] Issue with dev_type: type-PCI using openstack-mitaka release on host (Cent OS 7.2)

Hi Moshe,

Thank you for the clarification. I want to passthrough PCI device . Thus  I have configured the “device_type”: “type-PCI” in nova.conf. But  it shows “dev_type: type-PF” in MariaDB and nova.compute log. I think it should show “dev_type: type-PCI”. Am I missing anything ?
This will do what you called passthrough you PCI device is just indicated that your PCI has is actually Physical Function (you are passing (Ethernet controller) )
So Libvirt will do this:
<hostdev mode='subsystem' type='pci' managed='yes'>
  <source>
    <address domain='0x0000'
             bus='0x06'
             slot='0x12'
             function='0x5'/>
  </source>
</hostdev>

Regards,
Chinmaya

On Wed, Jun 8, 2016 at 10:58 AM, Moshe Levi <moshele at mellanox.com<mailto:moshele at mellanox.com>> wrote:


From: Chinmaya Dwibedy [mailto:ckdwibedy at gmail.com<mailto:ckdwibedy at gmail.com>]
Sent: Wednesday, June 08, 2016 7:43 AM
To: openstack at lists.openstack.org<mailto:openstack at lists.openstack.org>
Subject: [Openstack] Issue with dev_type: type-PCI using openstack-mitaka release on host (Cent OS 7.2)

Hi All,

I want the Ethernet Card to be used for PCI Passthrough device. Note that, I have installed openstack-mitaka release on host (Cent OS 7.2).

[root at localhost ~(keystone_admin)]# lspci -nn | grep 154d
07:00.0 Ethernet controller [0200]: Intel Corporation Ethernet 10G 2P X520 Adapter [8086:154d] (rev 01)
07:00.1 Ethernet controller [0200]: Intel Corporation Ethernet 10G 2P X520 Adapter [8086:154d] (rev 01)
[root at localhost ~(keystone_admin)]#

Here goes the configurations done in nova.conf

pci_alias={"vendor_id":"8086", "product_id":"154d", "device_type":"type-PCI", "name":"a1"}
pci_passthrough_whitelist =  [{"vendor_id":"8086","product_id":"154d"}]
scheduler_default_filters=RetryFilter,AvailabilityZoneFilter,RamFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,CoreFilter,PciPassthroughFilter



MariaDB [nova]> select pci_stats from compute_nodes;
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| pci_stats                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| {"nova_object.version": "1.1", "nova_object.changes": ["objects"], "nova_object.name<http://nova_object.name>": "PciDevicePoolList", "nova_object.data": {"objects": [{"nova_object.version": "1.1", "nova_object.changes": ["count", "numa_node", "vendor_id", "product_id", "tags"], "nova_object.name<http://nova_object.name>": "PciDevicePool", "nova_object.data": {"count": 2, "numa_node": 0, "vendor_id": "8086", "product_id": "154d", "tags": {"dev_type": "type-PF"}}, "nova_object.namespace": "nova"}]}, "nova_object.namespace": "nova"} |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

MariaDB [nova]> select count(*) from pci_devices where status="available" and deleted=0;
+----------+
| count(*) |
+----------+
|        2 |
+----------+
1 row in set (0.00 sec)

MariaDB [nova]>

MariaDB [nova]> SELECT * FROM nova.pci_devices;
+---------------------+---------------------+---------------------+---------+----+-----------------+--------------+------------+-----------+----------+------------------+-----------------+-----------+------------+---------------+------------+-----------+-------------+
| created_at          | updated_at          | deleted_at          | deleted | id | compute_node_id | address      | product_id | vendor_id | dev_type | dev_id           | label           | status    | extra_info | instance_uuid | request_id | numa_node | parent_addr |
+---------------------+---------------------+---------------------+---------+----+-----------------+--------------+------------+-----------+----------+------------------+-----------------+-----------+------------+---------------+------------+-----------+-------------+
| 2016-06-07 09:27:49 | 2016-06-08 04:24:46 | NULL                |       0 |  3 |               1 | 0000:07:00.0 | 154d       | 8086      | type-PF  | pci_0000_07_00_0 | label_8086_154d | available | {}         | NULL          | NULL       |         0 | NULL        |
| 2016-06-07 09:27:49 | 2016-06-08 04:24:46 | NULL                |       0 |  4 |               1 | 0000:07:00.1 | 154d       | 8086      | type-PF  | pci_0000_07_00_1 | label_8086_154d | available | {}         | NULL          | NULL       |         0 | NULL        |
+---------------------+---------------------+---------------------+---------+----+-----------------+--------------+------------+-----------+----------+------------------+-----------------+-----------+------------+---------------+------------+-----------+-------------+
4 rows in set (0.00 sec)

MariaDB [nova]>

There is a check in Nova ( get_device_type in https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py) that determines if a given PCI device is:
1)      A normal PCI device,
2)      An SR-IOV physical function (PF); or
3)      An SR-IOV virtual function (VF).
If it's a normal PCI device or a virtual function it's considered for passthrough, if it's a PF it's not (considered to be owned by the host/hypervisor).
No, you can passthrough PCI, PF and VF

What I understand, currently it is not possible to pass through a physical function to an OpenStack instance. Furthermore I have configured the “device_type”: “type-PCI” as stated above in nova.conf. Then why it shows “dev_type: type-PF” in MariaDB?
If you want to passthrough PF  you should change the pci_alias to "device_type":"type-PF"
Can anyone please let me know is it a bug in openstack-mitaka release or I have missed anything ? Please feel free to let me know if additional information is needed.

Regards,
Chinmaya

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20160608/944d6c21/attachment.html>