<div dir="ltr"><div><div><div>Hi Irena and Murali, <br><br>Thanks a lot for your reply!<br><br></div>Here is the output from pci_devices table of nova db:<br><br>select * from pci_devices;<br>+---------------------+------------+------------+---------+----+-----------------+--------------+------------+-----------+----------+------------------+-----------------+-----------+-----------------------------------+---------------+------------+<br>| created_at | updated_at | deleted_at | deleted | id | compute_node_id | address | product_id | vendor_id | dev_type | dev_id | label | status | extra_info | instance_uuid | request_id |<br>+---------------------+------------+------------+---------+----+-----------------+--------------+------------+-----------+----------+------------------+-----------------+-----------+-----------------------------------+---------------+------------+<br>| 2014-12-15 12:10:52 | NULL | NULL | 0 | 1 | 1 | 0000:03:10.0 | 10ed | 8086 | type-VF | pci_0000_03_10_0 | label_8086_10ed | available | {"phys_function": "0000:03:00.0"} | NULL | NULL |<br>| 2014-12-15 12:10:52 | NULL | NULL | 0 | 2 | 1 | 0000:03:10.2 | 10ed | 8086 | type-VF | pci_0000_03_10_2 | label_8086_10ed | available | {"phys_function": "0000:03:00.0"} | NULL | NULL |<br>| 2014-12-15 12:10:52 | NULL | NULL | 0 | 3 | 1 | 0000:03:10.4 | 10ed | 8086 | type-VF | pci_0000_03_10_4 | label_8086_10ed | available | {"phys_function": "0000:03:00.0"} | NULL | NULL |<br>| 2014-12-15 12:10:52 | NULL | NULL | 0 | 4 | 1 | 0000:03:10.6 | 10ed | 8086 | type-VF | pci_0000_03_10_6 | label_8086_10ed | available | {"phys_function": "0000:03:00.0"} | NULL | NULL |<br>| 2014-12-15 12:10:53 | NULL | NULL | 0 | 5 | 1 | 0000:03:10.1 | 10ed | 8086 | type-VF | pci_0000_03_10_1 | label_8086_10ed | available | {"phys_function": "0000:03:00.1"} | NULL | NULL |<br>| 2014-12-15 12:10:53 | NULL | NULL | 0 | 6 | 1 | 0000:03:10.3 | 10ed | 8086 | type-VF | pci_0000_03_10_3 | label_8086_10ed | available | {"phys_function": "0000:03:00.1"} | NULL | NULL |<br>| 2014-12-15 12:10:53 | NULL | NULL | 0 | 7 | 1 | 0000:03:10.5 | 10ed | 8086 | type-VF | pci_0000_03_10_5 | label_8086_10ed | available | {"phys_function": "0000:03:00.1"} | NULL | NULL |<br>| 2014-12-15 12:10:53 | NULL | NULL | 0 | 8 | 1 | 0000:03:10.7 | 10ed | 8086 | type-VF | pci_0000_03_10_7 | label_8086_10ed | available | {"phys_function": "0000:03:00.1"} | NULL | NULL |<br>+---------------------+------------+------------+---------+----+-----------------+--------------+------------+-----------+----------+------------------+-----------------+-----------+-----------------------------------+---------------+------------+<br><br></div>output from <span style="font-size:11pt;font-family:"Calibri","sans-serif";color:rgb(31,73,125)">select hypervisor_hostname,pci_stats from compute_nodes; is:<br>+---------------------+-------------------------------------------------------------------------------------------+<br>| hypervisor_hostname | pci_stats |<br>+---------------------+-------------------------------------------------------------------------------------------+<br>| blade08 | [{"count": 8, "vendor_id": "8086", "physical_network": "ext-net", "product_id": "10ed"}] |<br>+---------------------+-------------------------------------------------------------------------------------------+<br><br></span></div><span style="font-size:11pt;font-family:"Calibri","sans-serif";color:rgb(31,73,125)">Moreover, I have set </span>agent_required = True in <span style="color:rgb(0,0,0);font-family:Consolas,Menlo,Monaco,"Lucida Console","Liberation Mono","DejaVu Sans Mono","Bitstream Vera Sans Mono",monospace,serif;font-size:12px;line-height:21px">/etc/neutron/plugins/ml2/ml2_conf_sriov.ini. but still found no sriov agent running. <br># Defines configuration options for SRIOV NIC Switch MechanismDriver<br># and Agent<br><br>[ml2_sriov]<br># (ListOpt) Comma-separated list of<br># supported Vendor PCI Devices, in format vendor_id:product_id<br>#<br>#supported_pci_vendor_devs = 8086:10ca, 8086:10ed<br>supported_pci_vendor_devs = 8086:10ed<br># Example: supported_pci_vendor_devs = 15b3:1004<br>#<br># (BoolOpt) Requires running SRIOV neutron agent for port binding<br>agent_required = True<br><br>[sriov_nic]<br># (ListOpt) Comma-separated list of <physical_network>:<network_device><br># tuples mapping physical network names to the agent's node-specific<br># physical network device interfaces of SR-IOV physical function to be used<br># for VLAN networks. All physical networks listed in network_vlan_ranges on<br># the server should have mappings to appropriate interfaces on each agent.<br>#<br>physical_device_mappings = ext-net:br-ex<br># Example: physical_device_mappings = physnet1:eth1<br>#<br># (ListOpt) Comma-separated list of <network_device>:<vfs__to_exclude><br># tuples, mapping network_device to the agent's node-specific list of virtual<br># functions that should not be used for virtual networking.<br># vfs_to_exclude is a semicolon-separated list of virtual<br># functions to exclude from network_device. The network_device in the<br># mapping should appear in the physical_device_mappings list.<br># exclude_devices =<br># Example: exclude_devices = eth1:0000:07:00.2; 0000:07:00.3<br>========================================================================================<br>pci_passthrough_whitelist from /etc/nova/nova.conf:<br>pci_passthrough_whitelist = {"address":"*:03:10.*","physical_network":"ext-net"}<br></span><div><span style="font-size:11pt;font-family:"Calibri","sans-serif";color:rgb(31,73,125)">====================================================<br>/etc/neutron/plugins/ml2/ml2_conf.ini:<br><br>[ml2]<br># (ListOpt) List of network type driver entrypoints to be loaded from<br># the neutron.ml2.type_drivers namespace.<br>#<br>#type_drivers = local,flat,vlan,gre,vxlan<br>#Example: type_drivers = flat,vlan,gre,vxlan<br>#type_drivers = flat,gre, vlan<br>type_drivers = flat,vlan<br><br># (ListOpt) Ordered list of network_types to allocate as tenant<br># networks. The default value 'local' is useful for single-box testing<br># but provides no connectivity between hosts.<br>#<br># tenant_network_types = local<br># Example: tenant_network_types = vlan,gre,vxlan<br>#tenant_network_types = gre, vlan<br>tenant_network_types = vlan<br><br># (ListOpt) Ordered list of networking mechanism driver entrypoints<br># to be loaded from the neutron.ml2.mechanism_drivers namespace.<br>mechanism_drivers = openvswitch, sriovnicswitch<br># Example: mechanism_drivers = openvswitch,mlnx<br># Example: mechanism_drivers = arista<br># Example: mechanism_drivers = cisco,logger<br># Example: mechanism_drivers = openvswitch,brocade<br># Example: mechanism_drivers = linuxbridge,brocade<br><br># (ListOpt) Ordered list of extension driver entrypoints<br># to be loaded from the neutron.ml2.extension_drivers namespace.<br># extension_drivers =<br># Example: extension_drivers = anewextensiondriver<br><br>[ml2_type_flat]<br># (ListOpt) List of physical_network names with which flat networks<br># can be created. Use * to allow flat networks with arbitrary<br># physical_network names.<br>#<br>flat_networks = ext-net<br># Example:flat_networks = physnet1,physnet2<br># Example:flat_networks = *<br><br>[ml2_type_vlan]<br># (ListOpt) List of <physical_network>[:<vlan_min>:<vlan_max>] tuples<br># specifying physical_network names usable for VLAN provider and<br># tenant networks, as well as ranges of VLAN tags on each<br># physical_network available for allocation as tenant networks.<br>network_vlan_ranges = ext-net:2:100<br># Example: network_vlan_ranges = physnet1:1000:2999,physnet2<br><br>[ml2_type_gre]<br># (ListOpt) Comma-separated list of <tun_min>:<tun_max> tuples enumerating ranges of GRE tunnel IDs that are available for tenant network allocation<br>#tunnel_id_ranges = 1:1000<br><br>[ml2_type_vxlan]<br># (ListOpt) Comma-separated list of <vni_min>:<vni_max> tuples enumerating<br># ranges of VXLAN VNI IDs that are available for tenant network allocation.<br>#<br># vni_ranges =<br><br># (StrOpt) Multicast group for the VXLAN interface. When configured, will<br># enable sending all broadcast traffic to this multicast group. When left<br># unconfigured, will disable multicast VXLAN mode.<br>#<br># vxlan_group =<br># Example: vxlan_group = 239.1.1.1<br><br>[securitygroup]<br># Controls if neutron security group is enabled or not.<br># It should be false when you use nova security group.<br>enable_security_group = True<br><br># Use ipset to speed-up the iptables security groups. Enabling ipset support<br># requires that ipset is installed on L2 agent node.<br>enable_ipset = True<br>firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver<br>[ovs]<br>local_ip = controller<br>enable_tunneling = True<br>bridge_mappings = external:br-ex<br><br>[agent]<br>tunnel_types = vlan<br><br>[ml2_sriov]<br>agent_required = True<br><br></span><div>Please tell me what is wrong in there plus what exactly "physnet1" should be? Thanks again for all your help and suggestion....<br><br></div><div>Regards,<br></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Dec 16, 2014 at 10:42 AM, Irena Berezovsky <span dir="ltr"><<a href="mailto:irenab@mellanox.com" target="_blank">irenab@mellanox.com</a>></span> wrote:<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div link="blue" vlink="purple" lang="EN-US">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hi David,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">You error is not related to agent.
<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I would suggest to check:<u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><span>1.<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span dir="LTR"></span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> nova.conf at your compute node for pci whitelist configuration<u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><span>2.<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span dir="LTR"></span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Neutron server configuration for correct physical_network label matching the label in pci whitelist<u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><span>3.<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span dir="LTR"></span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Nova DB tables containing PCI devices entries:<u></u><u></u></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">"#echo 'use nova;select hypervisor_hostname,pci_stats from compute_nodes;' | mysql -u root"<u></u><u></u></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">You should not run SR-IOV agent in you setup. SR-IOV agent is an optional and currently does not add value if you use Intel NIC.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Regards,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Irena<u></u><u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> david jhon [mailto:<a href="mailto:djhon9813@gmail.com" target="_blank">djhon9813@gmail.com</a>]
<br>
<b>Sent:</b> Tuesday, December 16, 2014 5:54 AM<br>
<b>To:</b> Murali B<br>
<b>Cc:</b> OpenStack Development Mailing List (not for usage questions)<br>
<b>Subject:</b> Re: [openstack-dev] SRIOV-error<u></u><u></u></span></p><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Just to be more clear, command $lspci | grep -i Ethernet gives following output:<br>
<br>
01:00.0 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port Backplane Connection (rev 01)<br>
01:00.1 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port Backplane Connection (rev 01)<br>
03:00.0 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port Backplane Connection (rev 01)<br>
03:00.1 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port Backplane Connection (rev 01)<br>
03:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)<br>
03:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)<br>
03:10.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)<br>
03:10.3 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)<br>
03:10.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)<br>
03:10.5 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)<br>
03:10.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)<br>
03:10.7 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)<br>
04:00.0 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port Backplane Connection (rev 01)<br>
04:00.1 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port Backplane Connection (rev 01)<br>
04:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)<br>
04:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)<br>
04:10.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)<br>
04:10.3 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)<br>
04:10.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)<br>
04:10.5 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)<br>
04:10.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)<br>
04:10.7 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)<u></u><u></u></p>
</div>
<p class="MsoNormal">How can I make SR-IOV agent run and fix this bug?<u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Tue, Dec 16, 2014 at 8:36 AM, david jhon <<a href="mailto:djhon9813@gmail.com" target="_blank">djhon9813@gmail.com</a>> wrote:<u></u><u></u></p>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Hi Murali, <u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Thanks for your response, I did the same, it has resolved errors apparently but 1) neutron agent-list shows no agent for sriov, 2) neutron port is created successfully but creating vm is erred in scheduling
as follows:<br>
<br>
result from neutron agent-list:<br>
+--------------------------------------+--------------------+---------+-------+----------------+---------------------------+<br>
| id | agent_type | host | alive | admin_state_up | binary |<br>
+--------------------------------------+--------------------+---------+-------+----------------+---------------------------+<br>
| 2acc7044-e552-4601-b00b-00ba591b453f | Open vSwitch agent | blade08 | xxx | True | neutron-openvswitch-agent |<br>
| 595d07c6-120e-42ea-a950-6c77a6455f10 | Metadata agent | blade08 | :-) | True | neutron-metadata-agent |<br>
| a1f253a8-e02e-4498-8609-4e265285534b | DHCP agent | blade08 | :-) | True | neutron-dhcp-agent |<br>
| d46b29d8-4b5f-4838-bf25-b7925cb3e3a7 | L3 agent | blade08 | :-) | True | neutron-l3-agent |<br>
+--------------------------------------+--------------------+---------+-------+----------------+---------------------------+<br>
<br>
2014-12-15 19:30:44.546 40249 ERROR oslo.messaging.rpc.dispatcher [req-c7741cff-a7d8-422f-b605-6a1d976aeb09 ] Exception during message handling: PCI $<br>
2014-12-15 19:30:44.546 40249 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):<br>
2014-12-15 19:30:44.546 40249 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 13$<br>
2014-12-15 19:30:44.546 40249 TRACE oslo.messaging.rpc.dispatcher incoming.message))<br>
2014-12-15 19:30:44.546 40249 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 17$<br>
2014-12-15 19:30:44.546 40249 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args)<br>
2014-12-15 19:30:44.546 40249 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 12$<br>
2014-12-15 19:30:44.546 40249 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args)<br>
2014-12-15 19:30:44.546 40249 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/server.py", line 139, i$<br>
2014-12-15 19:30:44.546 40249 TRACE oslo.messaging.rpc.dispatcher return func(*args, **kwargs)<br>
2014-12-15 19:30:44.546 40249 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/scheduler/manager.py", line 175, in s$<br>
2014-12-15 19:30:44.546 40249 TRACE oslo.messaging.rpc.dispatcher filter_properties)<br>
2014-12-15 19:30:44.546 40249 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/scheduler/filter_scheduler.py", line $<br>
2014-12-15 19:30:44.546 40249 TRACE oslo.messaging.rpc.dispatcher filter_properties)<br>
2014-12-15 19:30:44.546 40249 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/scheduler/filter_scheduler.py", line $<br>
2014-12-15 19:30:44.546 40249 TRACE oslo.messaging.rpc.dispatcher chosen_host.obj.consume_from_instance(instance_properties)<br>
2014-12-15 19:30:44.546 40249 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/scheduler/host_manager.py", line 246,$<br>
2014-12-15 19:30:44.546 40249 TRACE oslo.messaging.rpc.dispatcher self.pci_stats.apply_requests(pci_requests.requests)<br>
2014-12-15 19:30:44.546 40249 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/pci/pci_stats.py", line 209, in apply$<br>
2014-12-15 19:30:44.546 40249 TRACE oslo.messaging.rpc.dispatcher raise exception.PciDeviceRequestFailed(requests=requests)<br>
2014-12-15 19:30:44.546 40249 TRACE oslo.messaging.rpc.dispatcher PciDeviceRequestFailed: PCI device request ({'requests': [InstancePCIRequest(alias_$<br>
2014-12-15 19:30:44.546 40249 TRACE oslo.messaging.rpc.dispatcher.<u></u><u></u></p>
</div>
<p class="MsoNormal">Moreover, no /var/log/sriov-agent.log file exists. Please help me to fix this issue. Thanks everyone!<u></u><u></u></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Mon, Dec 15, 2014 at 5:18 PM, Murali B <<a href="mailto:mbirru@gmail.com" target="_blank">mbirru@gmail.com</a>> wrote:<u></u><u></u></p>
<div>
<p class="MsoNormal">Hi David,<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Please add as per the Irena suggestion<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">FYI: refer the below configuration<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><a href="http://pastebin.com/DGmW7ZEg" target="_blank">http://pastebin.com/DGmW7ZEg</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Thanks<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#888888">-Murali<u></u><u></u></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div></div></div>
</div>
</blockquote></div></div>