<p dir="ltr">Try with the rtl8139 driver and try again</p>
<div class="gmail_quote">El 16/12/2014 20:09, "Georgios Dimitrakakis" <<a href="mailto:giorgis@acmac.uoc.gr">giorgis@acmac.uoc.gr</a>> escribió:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Changing<br>
<br>
gso on<br>
tso on<br>
gro off<br>
<br>
<br>
got me back to the initial status.<br>
<br>
<br>
Although now it starts with approximately 65-70MB/s  for a few seconds but then it drops down to 30MB/s<br>
<br>
Regards,<br>
<br>
George<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Try enabling the gso and the tso but keeping the gro disabled<br>
El 16/12/2014 19:38, "Georgios Dimitrakakis"  escribió:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I have changed that on both the node and the VMs and actually made<br>
things worse.<br>
<br>
I did that on both eth1 and br100 interfaces on the physical node.<br>
<br>
The transfer speed now is 15MB/s half of the one I had before!<br>
<br>
Have I missed something? I believe that this is not an expected<br>
behaviour?<br>
<br>
Here are the outputs for both br100 and eth1 on the node in case I<br>
have missed something:<br>
<br>
# ethtool -k br100<br>
Features for br100:<br>
rx-checksumming: off [fixed]<br>
tx-checksumming: on<br>
        tx-checksum-ipv4: off<br>
        tx-checksum-unneeded: off<br>
        tx-checksum-ip-generic: on<br>
        tx-checksum-ipv6: off<br>
        tx-checksum-fcoe-crc: off [fixed]<br>
        tx-checksum-sctp: off [fixed]<br>
scatter-gather: on<br>
        tx-scatter-gather: on<br>
        tx-scatter-gather-fraglist: on [fixed]<br>
tcp-segmentation-offload: off<br>
        tx-tcp-segmentation: off<br>
        tx-tcp-ecn-segmentation: on<br>
        tx-tcp6-segmentation: on<br>
udp-fragmentation-offload: on [fixed]<br>
generic-segmentation-offload: off<br>
generic-receive-offload: off<br>
large-receive-offload: off [fixed]<br>
rx-vlan-offload: off [fixed]<br>
tx-vlan-offload: off [fixed]<br>
ntuple-filters: off [fixed]<br>
receive-hashing: off [fixed]<br>
highdma: on [fixed]<br>
rx-vlan-filter: off [fixed]<br>
vlan-challenged: off [fixed]<br>
tx-lockless: on [fixed]<br>
netns-local: on [fixed]<br>
tx-gso-robust: off [fixed]<br>
tx-fcoe-segmentation: off [fixed]<br>
tx-gre-segmentation: on [fixed]<br>
tx-udp_tnl-segmentation: on [fixed]<br>
fcoe-mtu: off [fixed]<br>
loopback: off [fixed]<br>
<br>
# ethtool -k eth1<br>
Features for eth1:<br>
rx-checksumming: on<br>
tx-checksumming: on<br>
        tx-checksum-ipv4: off<br>
        tx-checksum-unneeded: off<br>
        tx-checksum-ip-generic: on<br>
        tx-checksum-ipv6: off<br>
        tx-checksum-fcoe-crc: off [fixed]<br>
        tx-checksum-sctp: off [fixed]<br>
scatter-gather: on<br>
        tx-scatter-gather: on<br>
        tx-scatter-gather-fraglist: off [fixed]<br>
tcp-segmentation-offload: off<br>
        tx-tcp-segmentation: off<br>
        tx-tcp-ecn-segmentation: off<br>
        tx-tcp6-segmentation: off<br>
udp-fragmentation-offload: off [fixed]<br>
generic-segmentation-offload: off<br>
generic-receive-offload: off<br>
large-receive-offload: off [fixed]<br>
rx-vlan-offload: on [fixed]<br>
tx-vlan-offload: on [fixed]<br>
ntuple-filters: off [fixed]<br>
receive-hashing: off [fixed]<br>
highdma: on [fixed]<br>
rx-vlan-filter: off [fixed]<br>
vlan-challenged: off [fixed]<br>
tx-lockless: off [fixed]<br>
netns-local: off [fixed]<br>
tx-gso-robust: off [fixed]<br>
tx-fcoe-segmentation: off [fixed]<br>
tx-gre-segmentation: off [fixed]<br>
tx-udp_tnl-segmentation: off [fixed]<br>
fcoe-mtu: off [fixed]<br>
loopback: off [fixed]<br>
<br>
Best,<br>
<br>
George<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Disabling it only on the nodes should boost the speed but<br>
disabling it<br>
in the vms too improves greatly the speed<br>
El 16/12/2014 19:13, "Georgios Dimitrakakis"  escribió:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Ooops...It seems that I have been confused..<br>
<br>
The pasted part is indeed from the node when I was looking<br>
somewhere else....<br>
<br>
Thanks a lot for noticing that Adrian!!!!<br>
<br>
I will turn it off on the nodes and test again!<br>
<br>
Should it be off on both the nodes and the VMs?<br>
<br>
Regards,<br>
<br>
George<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
That shows that those 3 offload settibgs are enabled.<br>
El 16/12/2014 19:01, "Georgios Dimitrakakis"  escribió:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I believe that they are already disabled.<br>
<br>
Here is the ethtool output:<br>
<br>
# ethtool --show-offload eth1<br>
Features for eth1:<br>
rx-checksumming: on<br>
tx-checksumming: on<br>
        tx-checksum-ipv4: off<br>
        tx-checksum-unneeded: off<br>
        tx-checksum-ip-generic: on<br>
        tx-checksum-ipv6: off<br>
        tx-checksum-fcoe-crc: off [fixed]<br>
        tx-checksum-sctp: off [fixed]<br>
scatter-gather: on<br>
        tx-scatter-gather: on<br>
        tx-scatter-gather-fraglist: off [fixed]<br>
tcp-segmentation-offload: on<br>
        tx-tcp-segmentation: on<br>
        tx-tcp-ecn-segmentation: off<br>
        tx-tcp6-segmentation: on<br>
udp-fragmentation-offload: off [fixed]<br>
generic-segmentation-offload: on<br>
generic-receive-offload: on<br>
large-receive-offload: off [fixed]<br>
rx-vlan-offload: on [fixed]<br>
tx-vlan-offload: on [fixed]<br>
ntuple-filters: off [fixed]<br>
receive-hashing: off [fixed]<br>
highdma: on [fixed]<br>
rx-vlan-filter: off [fixed]<br>
vlan-challenged: off [fixed]<br>
tx-lockless: off [fixed]<br>
netns-local: off [fixed]<br>
tx-gso-robust: off [fixed]<br>
tx-fcoe-segmentation: off [fixed]<br>
tx-gre-segmentation: off [fixed]<br>
tx-udp_tnl-segmentation: off [fixed]<br>
fcoe-mtu: off [fixed]<br>
loopback: off [fixed]<br>
<br>
Regards,<br>
<br>
George<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Disable offloading on the nodes with: ethtool -K<br>
interfaceName<br>
gro<br>
off<br>
gso off tso off<br>
<br>
And then try it again<br>
El 16/12/2014 18:36, "Georgios Dimitrakakis"  escribió:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi all!<br>
<br>
In my OpenStack installation (Icehouse and use nova<br>
legacy<br>
networking) the VMs are talking to each other over a<br>
1Gbps<br>
network<br>
link.<br>
<br>
My issue is that although file transfers between<br>
physical<br>
(hypervisor) nodes can saturate that link transfers<br>
between<br>
VMs<br>
reach very lower speeds e.g. 30MB/s (approx. 240Mbps).<br>
<br>
My tests are performed by scping a large image file<br>
(approx.<br>
4GB)<br>
between the nodes and between the VMs.<br>
<br>
I have updated my images to use e1000 nic driver but<br>
the<br>
results<br>
remain the same.<br>
<br>
What are any other limiting factors?<br>
<br>
Does it has to do with the disk driver I am using? Does<br>
it<br>
play<br>
significant role the filesystem of the hypervisor node?<br>
<br>
Any ideas on how to approach the saturation of the<br>
1Gbps<br>
link?<br>
<br>
Best regards,<br>
<br>
George<br>
<br>
______________________________<u></u>_________________<br>
Mailing list:<br>
</blockquote>
<br>
</blockquote>
<br>
</blockquote>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
[2]<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
[1]<br>
[1] [1]<br>
Post to     : <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a> [1] [2]<br>
[2] [2]<br>
Unsubscribe :<br>
</blockquote>
<br>
</blockquote>
<br>
</blockquote>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
[3]<br>
<br>
</blockquote>
<br>
</blockquote>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
[4]<br>
[4]<br>
[2] mailto:<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.<u></u>openstack.org</a> [4] [5] [5]<br>
[3]<br>
<br>
</blockquote>
<br>
</blockquote>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
[5]<br>
[6]<br>
[6]<br>
[4] mailto:<a href="mailto:giorgis@acmac.uoc.gr" target="_blank">giorgis@acmac.uoc.gr</a> [6] [7] [7]<br>
</blockquote></blockquote>
<br>
Links:<br>
------<br>
[1]<br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[7]<br>
[8]<br>
[2] mailto:<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.<u></u>openstack.org</a> [8] [9]<br>
[3]<br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[9]<br>
[10]<br>
[4]<br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[10]<br>
[11]<br>
[5] mailto:<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.<u></u>openstack.org</a> [11] [12]<br>
[6]<br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[12]<br>
[13]<br>
[7] mailto:<a href="mailto:giorgis@acmac.uoc.gr" target="_blank">giorgis@acmac.uoc.gr</a> [13] [14]<br>
[8] mailto:<a href="mailto:giorgis@acmac.uoc.gr" target="_blank">giorgis@acmac.uoc.gr</a> [14] [15]<br>
</blockquote></blockquote>
<br>
Links:<br>
------<br>
[1] <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[15]<br>
[2] mailto:<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.<u></u>openstack.org</a> [16]<br>
[3] <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[17]<br>
[4] <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[18]<br>
[5] mailto:<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.<u></u>openstack.org</a> [19]<br>
[6] <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[20]<br>
[7] mailto:<a href="mailto:giorgis@acmac.uoc.gr" target="_blank">giorgis@acmac.uoc.gr</a> [21]<br>
[8] <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[22]<br>
[9] mailto:<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.<u></u>openstack.org</a> [23]<br>
[10]<br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a> [24]<br>
[11]<br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a> [25]<br>
[12] mailto:<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.<u></u>openstack.org</a> [26]<br>
[13]<br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a> [27]<br>
[14] mailto:<a href="mailto:giorgis@acmac.uoc.gr" target="_blank">giorgis@acmac.uoc.gr</a> [28]<br>
[15] mailto:<a href="mailto:giorgis@acmac.uoc.gr" target="_blank">giorgis@acmac.uoc.gr</a> [29]<br>
[16] mailto:<a href="mailto:giorgis@acmac.uoc.gr" target="_blank">giorgis@acmac.uoc.gr</a> [30]<br>
</blockquote></blockquote>
<br>
<br>
Links:<br>
------<br>
[1] mailto:<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.<u></u>openstack.org</a><br>
[2] <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[3] <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[4] mailto:<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.<u></u>openstack.org</a><br>
[5] <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[6] mailto:<a href="mailto:giorgis@acmac.uoc.gr" target="_blank">giorgis@acmac.uoc.gr</a><br>
[7] <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[8] mailto:<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.<u></u>openstack.org</a><br>
[9] <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[10] <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[11] mailto:<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.<u></u>openstack.org</a><br>
[12] <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[13] mailto:<a href="mailto:giorgis@acmac.uoc.gr" target="_blank">giorgis@acmac.uoc.gr</a><br>
[14] mailto:<a href="mailto:giorgis@acmac.uoc.gr" target="_blank">giorgis@acmac.uoc.gr</a><br>
[15] <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[16] mailto:<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.<u></u>openstack.org</a><br>
[17] <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[18] <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[19] mailto:<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.<u></u>openstack.org</a><br>
[20] <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[21] mailto:<a href="mailto:giorgis@acmac.uoc.gr" target="_blank">giorgis@acmac.uoc.gr</a><br>
[22] <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[23] mailto:<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.<u></u>openstack.org</a><br>
[24] <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[25] <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[26] mailto:<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.<u></u>openstack.org</a><br>
[27] <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
[28] mailto:<a href="mailto:giorgis@acmac.uoc.gr" target="_blank">giorgis@acmac.uoc.gr</a><br>
[29] mailto:<a href="mailto:giorgis@acmac.uoc.gr" target="_blank">giorgis@acmac.uoc.gr</a><br>
[30] mailto:<a href="mailto:giorgis@acmac.uoc.gr" target="_blank">giorgis@acmac.uoc.gr</a><br>
[31] mailto:<a href="mailto:giorgis@acmac.uoc.gr" target="_blank">giorgis@acmac.uoc.gr</a><br>
</blockquote>
<br>
--<br>
</blockquote></div>