<!DOCTYPE html><html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8" /></head><body><div data-html-editor-font-wrapper="true" style="font-family: arial, sans-serif; font-size: 13px;">Hello everybody,<br><br>after a lot of investigations, I finally get rid of that Kernel skb_warn_bad_offload crash. To do so you want to disable completely GSO and even CSUM from the virtio_net module by adding [1] into your preferred modprobe.d file.<br><br>You probably want to know that ethtool -K ethernet_int tso off gso off whatever off isn't sufficient and doesn't override the default of virtio_net module, which is GSO and CSUM on.<br><br>Hope it helps someone, somewhere :)<br><br>See you,<br>Regards<br><br>[1] options virtio_net gso=0 and csum=0<br><br><br><br><br><br>May 20 2016 11:06 AM, "Leslie-Alexandre DENIS" <<a target="_blank" tabindex="-1" href="mailto:%22Leslie-Alexandre%20DENIS%22%20<contact@ladenis.fr>">contact@ladenis.fr</a>> wrote:<blockquote><div><div><div style="font-family: arial, sans-serif;font-size: 13px">Hello,<br><br>pretty hard problem today for me, as I need to build a Docker platform test inside OpenStack with CentOS 7.2 all around and apparently the Ethernet offloading features don't like virtio_net at all !<br>Not even one HTTP packet is going in/out through Docker bridges and host's NIC, the Kernel stack trace refers to WARNING: at net/core/dev.c:2263 skb_warn_bad_offload+0xcd/0xda().<br>I detect some bad TCP checksum in the VM for HTTP traffic but it must be related to offloading, which is normally disabled...<br>On the contrary a second test with FTP traffic seems to work.<br><br>I disabled all the interfaces offloading features via ethtool -K, more precisely rx off tx off tso off lro off gro off gso off. It does nothing regarding to the Docker problem.<br><br>The only way to run a container is to pass the --net host argument in order to not use the bridge networking, and it works.<br><br>So my question is, does anybody is using nested Docker Engine inside KVM ? Any hint ?<br><br>*facts*<br><br>* VM net args >> -netdev tap,fd=48,id=hostnet0,vhost=on,vhostfd=52 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:c4:2a:7e,bus=pci.0,addr=0x3<br>* VM qemu-kvm-ev-2.3.0-31.el7_2.4.1<br>* VM CentOS 7.2.1511 with 3.10.0-327.18.2.el7.x86_64 Kernel<br>* Docker version 1.11.1, build 5604cbe<br>* Docker bridge options (defaults):<br> "com.docker.network.bridge.default_bridge": "true",<br> "com.docker.network.bridge.enable_icc": "true",<br> "com.docker.network.bridge.enable_ip_masquerade": "true",<br> "com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",<br> "com.docker.network.bridge.name": "docker0",<br> "com.docker.network.driver.mtu": "1500"<br>* Kernel TCP/IP params ok<br>* Offloading turned off via ethtool -K ... rx off tx off tso off lro off gro off gso off<br>* Kernel stack trace ><br><br>May 20 10:54:08 ccosvms0094 kernel: Call Trace:<br>May 20 10:54:08 ccosvms0094 kernel: <IRQ> [<ffffffff81635a0c>] dump_stack+0x19/0x1b<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff8107b200>] warn_slowpath_common+0x70/0xb0<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff8107b29c>] warn_slowpath_fmt+0x5c/0x80<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff812f96a3>] ? ___ratelimit+0x93/0x100<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff81638764>] skb_warn_bad_offload+0xcd/0xda<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff8152c4e9>] __skb_gso_segment+0x79/0xb0<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff8152c865>] validate_xmit_skb.part.86+0x135/0x2f0<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff8152d2ad>] dev_queue_xmit+0x4dd/0x570<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffffa04e1306>] ? ipv4_confirm+0x86/0x100 [nf_conntrack_ipv4]<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff8156b3bd>] ip_finish_output+0x53d/0x7d0<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff8156c36f>] ip_output+0x6f/0xe0<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff8156ae80>] ? ip_fragment+0x8b0/0x8b0<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff81567da6>] ip_forward_finish+0x66/0x80<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff81568137>] ip_forward+0x377/0x490<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff81567d40>] ? ip_frag_mem+0x40/0x40<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff81565e6d>] ip_rcv_finish+0x7d/0x350<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff81566806>] ip_rcv+0x2b6/0x410<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff81565df0>] ? inet_del_offload+0x40/0x40<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff8152ada2>] __netif_receive_skb_core+0x582/0x7d0<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff8152b008>] __netif_receive_skb+0x18/0x60<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff8152b090>] netif_receive_skb+0x40/0xc0<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffffa0179b48>] virtnet_poll+0x3d8/0x700 [virtio_net]<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff8152b4c2>] net_rx_action+0x152/0x240<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff81084b0f>] __do_softirq+0xef/0x280<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff81647adc>] call_softirq+0x1c/0x30<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff81016fc5>] do_softirq+0x65/0xa0<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff81084ea5>] irq_exit+0x115/0x120<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff81648678>] do_IRQ+0x58/0xf0<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff8163d92d>] common_interrupt+0x6d/0x6d<br>May 20 10:54:08 ccosvms0094 kernel: <EOI> [<ffffffff81058e96>] ? native_safe_halt+0x6/0x10<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff8101dbcf>] default_idle+0x1f/0xc0<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff8101e4d6>] arch_cpu_idle+0x26/0x30<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff810d6445>] cpu_startup_entry+0x245/0x290<br>May 20 10:54:08 ccosvms0094 kernel: [<ffffffff810475fa>] start_secondary+0x1ba/0x230<br>May 20 10:54:08 ccosvms0094 kernel: ---[ end trace b2c6008796d4cce2 ]---<br><br><br><br>Thanks a lot,<br>Leslie<br> </div></div></div></blockquote></div></body></html>