<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">在 2015年03月11日 22:15, jacob jacob 写道:<br>
Hi, jacob<br>
<br>
we now find przemyslaw.czesnowicz have same NIC, hope will
help a little bit. <br>
<br>
Yongli He<br>
</div>
<blockquote
cite="mid:CA+AkT2hSyU=cX=Ek=YhuNEyeTAnsLQH_61267LMsQM_pK4j-Lg@mail.gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<div dir="ltr"><br>
<div class="gmail_quote">---------- Forwarded message ----------<br>
From: <b class="gmail_sendername">jacob jacob</b> <span
dir="ltr"><<a moz-do-not-send="true"
href="mailto:opstkusr@gmail.com">opstkusr@gmail.com</a>></span><br>
Date: Tue, Mar 10, 2015 at 6:00 PM<br>
Subject: PCI passthrough of 40G ethernet interface<br>
To: <a moz-do-not-send="true"
href="mailto:openstack@lists.openstack.org">openstack@lists.openstack.org</a><br>
<br>
<br>
<div dir="ltr"><br>
<div class="gmail_quote">
<div dir="ltr"><span class=""><span style="font-size:13px">Hi,</span>
<div style="font-size:13px">I'm interested in finding
out if anyone has successfully tested PCI
passthrough functionality for 40G interfaces in
Openstack(KVM hypervisor).</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">I am trying this out on a
Fedora 21 host with Fedora 21 VM
image.(3.18.7-200.fc21.x86_64)</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">Was able to successfully
test PCI passthrough of 10 G interfaces:</div>
<div style="font-size:13px"> Ethernet controller:
Intel Corporation 82599ES 10-Gigabit SFI/SFP+
Network Connection (rev 01)</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">With 40G interface
testing, the PCI device is passed through to the VM
but data transfer is failing.</div>
<div style="font-size:13px"> 0a:00.1 Ethernet
controller: Intel Corporation Ethernet Controller
XL710 for 40GbE QSFP+ (rev 01)</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">Tried this with both the
i40e driver and latest dpdk driver but no luck so
far.</div>
<div style="font-size:13px"><br>
</div>
</span>
<div style="font-size:13px">With the i40e driver, the
data transfer fails.</div>
<div>
<div class="h5">
<div style="font-size:13px"> Relevant dmesg output:</div>
<div style="font-size:13px"> [ 11.544088] i40e
0000:00:05.0 eth1: NIC Link is Up 40 Gbps Full
Duplex, Flow Control: None</div>
<div style="font-size:13px">[ 11.689178] i40e
0000:00:06.0 eth2: NIC Link is Up 40 Gbps Full
Duplex, Flow Control: None</div>
<div style="font-size:13px">[ 16.704071]
------------[ cut here ]------------</div>
<div style="font-size:13px">[ 16.705053] WARNING:
CPU: 1 PID: 0 at net/sched/sch_generic.c:303
dev_watchdog+0x23e/0x250()</div>
<div style="font-size:13px">[ 16.705053] NETDEV
WATCHDOG: eth1 (i40e): transmit queue 1 timed out</div>
<div style="font-size:13px">[ 16.705053] Modules
linked in: cirrus ttm drm_kms_helper i40e drm
ppdev serio_raw i2c_piix4 virtio_net parport_pc
ptp virtio_balloon crct10dif_pclmul pps_core
parport pvpanic crc32_pclmul ghash_clmulni_intel
virtio_blk crc32c_intel virtio_pci virtio_ring
virtio ata_generic pata_acpi</div>
<div style="font-size:13px">[ 16.705053] CPU: 1
PID: 0 Comm: swapper/1 Not tainted
3.18.7-200.fc21.x86_64 #1</div>
<div style="font-size:13px">[ 16.705053] Hardware
name: Fedora Project OpenStack Nova, BIOS
1.7.5-20140709_153950- 04/01/2014</div>
<div style="font-size:13px">[ 16.705053]
0000000000000000 2e5932b294d0c473
ffff88043fc83d48 ffffffff8175e686</div>
<div style="font-size:13px">[ 16.705053]
0000000000000000 ffff88043fc83da0
ffff88043fc83d88 ffffffff810991d1</div>
<div style="font-size:13px">[ 16.705053]
ffff88042958f5c0 0000000000000001
ffff88042865f000 0000000000000001</div>
<div style="font-size:13px">[ 16.705053] Call
Trace:</div>
<div style="font-size:13px">[ 16.705053]
<IRQ> [<ffffffff8175e686>]
dump_stack+0x46/0x58</div>
<div style="font-size:13px">[ 16.705053]
[<ffffffff810991d1>]
warn_slowpath_common+0x81/0xa0</div>
<div style="font-size:13px">[ 16.705053]
[<ffffffff81099245>]
warn_slowpath_fmt+0x55/0x70</div>
<div style="font-size:13px">[ 16.705053]
[<ffffffff8166e62e>]
dev_watchdog+0x23e/0x250</div>
<div style="font-size:13px">[ 16.705053]
[<ffffffff8166e3f0>] ?
dev_graft_qdisc+0x80/0x80</div>
<div style="font-size:13px">[ 16.705053]
[<ffffffff810fd52a>]
call_timer_fn+0x3a/0x120</div>
<div style="font-size:13px">[ 16.705053]
[<ffffffff8166e3f0>] ?
dev_graft_qdisc+0x80/0x80</div>
<div style="font-size:13px">[ 16.705053]
[<ffffffff810ff692>]
run_timer_softirq+0x212/0x2f0</div>
<div style="font-size:13px">[ 16.705053]
[<ffffffff8109d7a4>]
__do_softirq+0x124/0x2d0</div>
<div style="font-size:13px">[ 16.705053]
[<ffffffff8109db75>] irq_exit+0x125/0x130</div>
<div style="font-size:13px">[ 16.705053]
[<ffffffff817681d8>]
smp_apic_timer_interrupt+0x48/0x60</div>
<div style="font-size:13px">[ 16.705053]
[<ffffffff817662bd>]
apic_timer_interrupt+0x6d/0x80</div>
<div style="font-size:13px">[ 16.705053]
<EOI> [<ffffffff811005c8>] ?
hrtimer_start+0x18/0x20</div>
<div style="font-size:13px">[ 16.705053]
[<ffffffff8105ca96>] ?
native_safe_halt+0x6/0x10</div>
<div style="font-size:13px">[ 16.705053]
[<ffffffff810f81d3>] ?
rcu_eqs_enter+0xa3/0xb0</div>
<div style="font-size:13px">[ 16.705053]
[<ffffffff8101ec7f>] default_idle+0x1f/0xc0</div>
<div style="font-size:13px">[ 16.705053]
[<ffffffff8101f64f>] arch_cpu_idle+0xf/0x20</div>
<div style="font-size:13px">[ 16.705053]
[<ffffffff810dad35>]
cpu_startup_entry+0x3c5/0x410</div>
<div style="font-size:13px">[ 16.705053]
[<ffffffff8104a2af>]
start_secondary+0x1af/0x1f0</div>
<div style="font-size:13px">[ 16.705053] ---[ end
trace 7bda53aeda558267 ]---</div>
<div style="font-size:13px">[ 16.705053] i40e
0000:00:05.0 eth1: tx_timeout recovery level 1</div>
<div style="font-size:13px">[ 16.705053] i40e
0000:00:05.0: i40e_vsi_control_tx: VSI seid 519 Tx
ring 0 disable timeout</div>
<div style="font-size:13px">[ 16.744198] i40e
0000:00:05.0: i40e_vsi_control_tx: VSI seid 520 Tx
ring 64 disable timeout</div>
<div style="font-size:13px">[ 16.779322] i40e
0000:00:05.0: i40e_ptp_init: added PHC on eth1</div>
<div style="font-size:13px">[ 16.791819] i40e
0000:00:05.0: PF 40 attempted to control timestamp
mode on port 1, which is owned by PF 1</div>
<div style="font-size:13px">[ 16.933869] i40e
0000:00:05.0 eth1: NIC Link is Up 40 Gbps Full
Duplex, Flow Control: None</div>
<div style="font-size:13px">[ 18.853624] SELinux:
initialized (dev tmpfs, type tmpfs), uses
transition SIDs</div>
<div style="font-size:13px">[ 22.720083] i40e
0000:00:05.0 eth1: tx_timeout recovery level 2</div>
<div style="font-size:13px">[ 22.826993] i40e
0000:00:05.0: i40e_vsi_control_tx: VSI seid 519 Tx
ring 0 disable timeout</div>
<div style="font-size:13px">[ 22.935288] i40e
0000:00:05.0: i40e_vsi_control_tx: VSI seid 520 Tx
ring 64 disable timeout</div>
<div style="font-size:13px">[ 23.669555] i40e
0000:00:05.0: i40e_ptp_init: added PHC on eth1</div>
<div style="font-size:13px">[ 23.682067] i40e
0000:00:05.0: PF 40 attempted to control timestamp
mode on port 1, which is owned by PF 1</div>
<div style="font-size:13px">[ 23.722423] i40e
0000:00:05.0 eth1: NIC Link is Up 40 Gbps Full
Duplex, Flow Control: None</div>
<div style="font-size:13px">[ 23.800206] i40e
0000:00:06.0: i40e_ptp_init: added PHC on eth2</div>
<div style="font-size:13px">[ 23.813804] i40e
0000:00:06.0: PF 48 attempted to control timestamp
mode on port 0, which is owned by PF 0</div>
<div style="font-size:13px">[ 23.855275] i40e
0000:00:06.0 eth2: NIC Link is Up 40 Gbps Full
Duplex, Flow Control: None</div>
<div style="font-size:13px">[ 38.720091] i40e
0000:00:05.0 eth1: tx_timeout recovery level 3</div>
<div style="font-size:13px">[ 38.725844] random:
nonblocking pool is initialized</div>
<div style="font-size:13px">[ 38.729874] i40e
0000:00:06.0: HMC error interrupt</div>
<div style="font-size:13px">[ 38.733425] i40e
0000:00:06.0: i40e_vsi_control_tx: VSI seid 518 Tx
ring 0 disable timeout</div>
<div style="font-size:13px">[ 38.738886] i40e
0000:00:06.0: i40e_vsi_control_tx: VSI seid 521 Tx
ring 64 disable timeout</div>
<div style="font-size:13px">[ 39.689569] i40e
0000:00:06.0: i40e_ptp_init: added PHC on eth2</div>
<div style="font-size:13px">[ 39.704197] i40e
0000:00:06.0: PF 48 attempted to control timestamp
mode on port 0, which is owned by PF 0</div>
<div style="font-size:13px">[ 39.746879] i40e
0000:00:06.0 eth2: NIC Link is Down</div>
<div style="font-size:13px">[ 39.838356] i40e
0000:00:05.0: i40e_ptp_init: added PHC on eth1</div>
<div style="font-size:13px">[ 39.851788] i40e
0000:00:05.0: PF 40 attempted to control timestamp
mode on port 1, which is owned by PF 1</div>
<div style="font-size:13px">[ 39.892822] i40e
0000:00:05.0 eth1: NIC Link is Down</div>
<div style="font-size:13px">[ 43.011610] i40e
0000:00:06.0 eth2: NIC Link is Up 40 Gbps Full
Duplex, Flow Control: None</div>
<div style="font-size:13px">[ 43.059976] i40e
0000:00:05.0 eth1: NIC Link is Up 40 Gbps Full
Duplex, Flow Control: None</div>
<div><br>
</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">Similarly, with dpdk
driver, no packet transfer is happening.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">The host was booted
with iommu=pt intel_iommu=on.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">It is to be noted that
everything works fine on the host itself. The
issue is seen only when the devices are passed
through to the VM.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">Would really appreciate
any additional information on how to address or
further debug this issue.</div>
<div style="font-size:13px"><br>
</div>
<div style="font-size:13px">Thanks</div>
</div>
</div>
</div>
<span class="HOEnZb"><font color="#888888">
</font></span></div>
<span class="HOEnZb"><font color="#888888">Jacob</font></span></div>
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: <a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<br>
</body>
</html>