Open Stack

Hello,

I am trying to configure Openstack Ocata with support for SR-IOV on Intel X710 NICs, but am struggling to launch any VMs.

I've followed the steps available at: https://docs.openstack.org/ocata/networking-guide/config-sriov.html, but am encountering an issue where my available hosts are filtered.


#####
Problem
#####

Launching a VM, configured using a port-id that uses "--binding:vnic_type direct" fails to launch.

#####
Details
#####

Following the guide available at

      https://docs.openstack.org/ocata/networking-guide/config-sriov.html

I have tried to enable the use of Virtual Function (VF) ports using SR-IOV on an on Intel X710 NIC. When I come to spin-up a VM using the steps documented, e.g.

      openstack server create --flavor m1.large --image cirros --nic port-id=$port_id test-sriov

the instantiation fails. In /var/log/nova-conductor.log, I can see: -

      2018-01-05 18:49:04.470 36491 WARNING nova.scheduler.utils [req-788817bb-4e78-4a1b-8bf5-240a20472cd8 0218d276a04e40ab806d4075671ef01b dd2323e3ab4f45f29b6e7840fb7c0d2c - - -] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available.
      Traceback (most recent call last):

      File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 218, in inner
        return func(*args, **kwargs)

     File "/usr/lib/python2.7/dist-packages/nova/scheduler/manager.py", line 98, in select_destinations
        dests = self.driver.select_destinations(ctxt, spec_obj)

     File "/usr/lib/python2.7/dist-packages/nova/scheduler/filter_scheduler.py", line 79, in select_destinations
        raise exception.NoValidHost(reason=reason)

      NoValidHost: No valid host was found. There are not enough hosts available.

               2018-01-05 18:49:04.471 36491 WARNING nova.scheduler.utils [req-788817bb-4e78-4a1b-8bf5-240a20472cd8 0218d276a04e40ab806d4075671ef01b dd2323e3ab4f45f29b6e7840fb7c0d2c - - -] [instance: f18454bb-ff7e-4171-8ebd-982197e353f7] Setting instance to ERROR state.

Note: NoValidHost: No valid host was found. There are not enough hosts available.

In /var/log/nova-scheduler.log I also see the following: -

               2018-01-05 18:49:04.371 37065 INFO nova.filters [...] Filter PciPassthroughFilter returned 0 hosts
      2018-01-05 18:49:04.372 37065 INFO nova.filters [...] Filtering removed all hosts for the request with instance ID 'f18454bb-ff7e-4171-8ebd-982197e353f7'. Filter results: ['PciPassthroughFilter: (start: 2, end: 0)']

Initially, I felt this might be because the VM flavor used didn't have a "pci_passthrough:alias" value set. I've made sure to set my flavor like so: -

      +----------------------------+--------------------------------------------------------------------+
      | Field                      | Value                                                              |
      +----------------------------+--------------------------------------------------------------------+
      | OS-FLV-DISABLED:disabled   | False                                                              |
      | OS-FLV-EXT-DATA:ephemeral  | 0                                                                  |
      | access_project_ids         | af8fadc445164950a6d7ad4f91e1e06b, dd2323e3ab4f45f29b6e7840fb7c0d2c |
      | disk                       | 5                                                                  |
      | id                         | fc76fed6-0583-4c6b-908b-3c49d6cd6652                               |
      | name                       | m2.mini                                                            |
      | os-flavor-access:is_public | False                                                              |
      | properties                 | pci_passthrough:alias='a1:1'                                       |
      | ram                        | 512                                                                |
      | rxtx_factor                | 1.0                                                                |
      | swap                       |                                                                    |
      | vcpus                      | 1                                                                  |
      +----------------------------+--------------------------------------------------------------------+

where "alias" is set in /etc/nova/nova.conf on the controller as: -

      [pci]
      alias = { "vendor_id":"8086", "product_id":"154c", "device_type":"type-VF","name":"a1" }

I've checked the PCI details on my VF's, and they are as follows: -


      81:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ [8086:1572] (rev 01)
      81:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ [8086:1572] (rev 01)
      81:02.0 Ethernet controller [0200]: Intel Corporation XL710/X710 Virtual Function [8086:154c] (rev 01)
      81:02.1 Ethernet controller [0200]: Intel Corporation XL710/X710 Virtual Function [8086:154c] (rev 01)
      81:02.2 Ethernet controller [0200]: Intel Corporation XL710/X710 Virtual Function [8086:154c] (rev 01)
      [and so on... 30x VFs in total]

As far as other configuration files look,

/etc/nova/nova.conf on the Compute is configured as so: -

      [pci]
      passthrough_whitelist ={ "devname": "enp129s0f0", "physical_network": "physnet2"}

the /etc/neutron/plugins/ml2/sriov_agent.ini on the Compute is configured to map my physical network to the NIC as follows: -

      [sriov_nic]
      physical_device_mappings =physnet2:enp129s0f0

and firewall disabled with: -

      [securitygroup]
      firewall_driver = neutron.agent.firewall.NoopFirewallDriver

/etc/neutron/plugins/ml2/ml2_conf.ini on the Controller has mechanism_drivers configures as: -

               mechanism_drivers = linuxbridge,sriovnicswitch


And lastly, /etc/nova/nova.conf on the Controller as the following filters set: -

               [DEFAULT]
      scheduler_default_filters =PciPassthroughFilter
      scheduler_available_filters = nova.scheduler.filters.all_filters

I've been investigating this for some time now, and am able to confirm that without pursuing the SR-IOV route I am able to create VLAN Provider networks on my NIC interface quite happily, and VMs can connect to them with no issue. Are there any obvious debugging routes I should investigate to try and resolve this?

Thanks for replies!

Ben

Ben Thomas
Senior Research Associate
UK 5G Testbeds and Trials - University of Bristol
Level 0 - Merchant Venturers Building
Woodland Road, Clifton, Bristol, BS8 1UB, United Kingdom
Email: ben.thomas at bristol.ac.uk
Smart Internet Lab<http://www.bristol.ac.uk/smart>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20180105/45c8c72c/attachment.html>