[Openstack] Network speed issue
Adrián Norte Fernández
adrian at bashlines.com
Tue Dec 16 19:20:27 UTC 2014
Try with the rtl8139 driver and try again
El 16/12/2014 20:09, "Georgios Dimitrakakis" <giorgis at acmac.uoc.gr>
escribió:
> Changing
>
> gso on
> tso on
> gro off
>
>
> got me back to the initial status.
>
>
> Although now it starts with approximately 65-70MB/s for a few seconds but
> then it drops down to 30MB/s
>
> Regards,
>
> George
>
> Try enabling the gso and the tso but keeping the gro disabled
>> El 16/12/2014 19:38, "Georgios Dimitrakakis" escribió:
>>
>> I have changed that on both the node and the VMs and actually made
>>> things worse.
>>>
>>> I did that on both eth1 and br100 interfaces on the physical node.
>>>
>>> The transfer speed now is 15MB/s half of the one I had before!
>>>
>>> Have I missed something? I believe that this is not an expected
>>> behaviour?
>>>
>>> Here are the outputs for both br100 and eth1 on the node in case I
>>> have missed something:
>>>
>>> # ethtool -k br100
>>> Features for br100:
>>> rx-checksumming: off [fixed]
>>> tx-checksumming: on
>>> tx-checksum-ipv4: off
>>> tx-checksum-unneeded: off
>>> tx-checksum-ip-generic: on
>>> tx-checksum-ipv6: off
>>> tx-checksum-fcoe-crc: off [fixed]
>>> tx-checksum-sctp: off [fixed]
>>> scatter-gather: on
>>> tx-scatter-gather: on
>>> tx-scatter-gather-fraglist: on [fixed]
>>> tcp-segmentation-offload: off
>>> tx-tcp-segmentation: off
>>> tx-tcp-ecn-segmentation: on
>>> tx-tcp6-segmentation: on
>>> udp-fragmentation-offload: on [fixed]
>>> generic-segmentation-offload: off
>>> generic-receive-offload: off
>>> large-receive-offload: off [fixed]
>>> rx-vlan-offload: off [fixed]
>>> tx-vlan-offload: off [fixed]
>>> ntuple-filters: off [fixed]
>>> receive-hashing: off [fixed]
>>> highdma: on [fixed]
>>> rx-vlan-filter: off [fixed]
>>> vlan-challenged: off [fixed]
>>> tx-lockless: on [fixed]
>>> netns-local: on [fixed]
>>> tx-gso-robust: off [fixed]
>>> tx-fcoe-segmentation: off [fixed]
>>> tx-gre-segmentation: on [fixed]
>>> tx-udp_tnl-segmentation: on [fixed]
>>> fcoe-mtu: off [fixed]
>>> loopback: off [fixed]
>>>
>>> # ethtool -k eth1
>>> Features for eth1:
>>> rx-checksumming: on
>>> tx-checksumming: on
>>> tx-checksum-ipv4: off
>>> tx-checksum-unneeded: off
>>> tx-checksum-ip-generic: on
>>> tx-checksum-ipv6: off
>>> tx-checksum-fcoe-crc: off [fixed]
>>> tx-checksum-sctp: off [fixed]
>>> scatter-gather: on
>>> tx-scatter-gather: on
>>> tx-scatter-gather-fraglist: off [fixed]
>>> tcp-segmentation-offload: off
>>> tx-tcp-segmentation: off
>>> tx-tcp-ecn-segmentation: off
>>> tx-tcp6-segmentation: off
>>> udp-fragmentation-offload: off [fixed]
>>> generic-segmentation-offload: off
>>> generic-receive-offload: off
>>> large-receive-offload: off [fixed]
>>> rx-vlan-offload: on [fixed]
>>> tx-vlan-offload: on [fixed]
>>> ntuple-filters: off [fixed]
>>> receive-hashing: off [fixed]
>>> highdma: on [fixed]
>>> rx-vlan-filter: off [fixed]
>>> vlan-challenged: off [fixed]
>>> tx-lockless: off [fixed]
>>> netns-local: off [fixed]
>>> tx-gso-robust: off [fixed]
>>> tx-fcoe-segmentation: off [fixed]
>>> tx-gre-segmentation: off [fixed]
>>> tx-udp_tnl-segmentation: off [fixed]
>>> fcoe-mtu: off [fixed]
>>> loopback: off [fixed]
>>>
>>> Best,
>>>
>>> George
>>>
>>> Disabling it only on the nodes should boost the speed but
>>>> disabling it
>>>> in the vms too improves greatly the speed
>>>> El 16/12/2014 19:13, "Georgios Dimitrakakis" escribió:
>>>>
>>>> Ooops...It seems that I have been confused..
>>>>>
>>>>> The pasted part is indeed from the node when I was looking
>>>>> somewhere else....
>>>>>
>>>>> Thanks a lot for noticing that Adrian!!!!
>>>>>
>>>>> I will turn it off on the nodes and test again!
>>>>>
>>>>> Should it be off on both the nodes and the VMs?
>>>>>
>>>>> Regards,
>>>>>
>>>>> George
>>>>>
>>>>> That shows that those 3 offload settibgs are enabled.
>>>>>> El 16/12/2014 19:01, "Georgios Dimitrakakis" escribió:
>>>>>>
>>>>>> I believe that they are already disabled.
>>>>>>>
>>>>>>> Here is the ethtool output:
>>>>>>>
>>>>>>> # ethtool --show-offload eth1
>>>>>>> Features for eth1:
>>>>>>> rx-checksumming: on
>>>>>>> tx-checksumming: on
>>>>>>> tx-checksum-ipv4: off
>>>>>>> tx-checksum-unneeded: off
>>>>>>> tx-checksum-ip-generic: on
>>>>>>> tx-checksum-ipv6: off
>>>>>>> tx-checksum-fcoe-crc: off [fixed]
>>>>>>> tx-checksum-sctp: off [fixed]
>>>>>>> scatter-gather: on
>>>>>>> tx-scatter-gather: on
>>>>>>> tx-scatter-gather-fraglist: off [fixed]
>>>>>>> tcp-segmentation-offload: on
>>>>>>> tx-tcp-segmentation: on
>>>>>>> tx-tcp-ecn-segmentation: off
>>>>>>> tx-tcp6-segmentation: on
>>>>>>> udp-fragmentation-offload: off [fixed]
>>>>>>> generic-segmentation-offload: on
>>>>>>> generic-receive-offload: on
>>>>>>> large-receive-offload: off [fixed]
>>>>>>> rx-vlan-offload: on [fixed]
>>>>>>> tx-vlan-offload: on [fixed]
>>>>>>> ntuple-filters: off [fixed]
>>>>>>> receive-hashing: off [fixed]
>>>>>>> highdma: on [fixed]
>>>>>>> rx-vlan-filter: off [fixed]
>>>>>>> vlan-challenged: off [fixed]
>>>>>>> tx-lockless: off [fixed]
>>>>>>> netns-local: off [fixed]
>>>>>>> tx-gso-robust: off [fixed]
>>>>>>> tx-fcoe-segmentation: off [fixed]
>>>>>>> tx-gre-segmentation: off [fixed]
>>>>>>> tx-udp_tnl-segmentation: off [fixed]
>>>>>>> fcoe-mtu: off [fixed]
>>>>>>> loopback: off [fixed]
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> George
>>>>>>>
>>>>>>> Disable offloading on the nodes with: ethtool -K
>>>>>>>> interfaceName
>>>>>>>> gro
>>>>>>>> off
>>>>>>>> gso off tso off
>>>>>>>>
>>>>>>>> And then try it again
>>>>>>>> El 16/12/2014 18:36, "Georgios Dimitrakakis" escribió:
>>>>>>>>
>>>>>>>> Hi all!
>>>>>>>>>
>>>>>>>>> In my OpenStack installation (Icehouse and use nova
>>>>>>>>> legacy
>>>>>>>>> networking) the VMs are talking to each other over a
>>>>>>>>> 1Gbps
>>>>>>>>> network
>>>>>>>>> link.
>>>>>>>>>
>>>>>>>>> My issue is that although file transfers between
>>>>>>>>> physical
>>>>>>>>> (hypervisor) nodes can saturate that link transfers
>>>>>>>>> between
>>>>>>>>> VMs
>>>>>>>>> reach very lower speeds e.g. 30MB/s (approx. 240Mbps).
>>>>>>>>>
>>>>>>>>> My tests are performed by scping a large image file
>>>>>>>>> (approx.
>>>>>>>>> 4GB)
>>>>>>>>> between the nodes and between the VMs.
>>>>>>>>>
>>>>>>>>> I have updated my images to use e1000 nic driver but
>>>>>>>>> the
>>>>>>>>> results
>>>>>>>>> remain the same.
>>>>>>>>>
>>>>>>>>> What are any other limiting factors?
>>>>>>>>>
>>>>>>>>> Does it has to do with the disk driver I am using? Does
>>>>>>>>> it
>>>>>>>>> play
>>>>>>>>> significant role the filesystem of the hypervisor node?
>>>>>>>>>
>>>>>>>>> Any ideas on how to approach the saturation of the
>>>>>>>>> 1Gbps
>>>>>>>>> link?
>>>>>>>>>
>>>>>>>>> Best regards,
>>>>>>>>>
>>>>>>>>> George
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Mailing list:
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>>>
>>>>>>> [2]
>>>>>>>>
>>>>>>>> [1]
>>>>>>>>> [1] [1]
>>>>>>>>> Post to : openstack at lists.openstack.org [1] [2]
>>>>>>>>> [2] [2]
>>>>>>>>> Unsubscribe :
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>>>
>>>>>>> [3]
>>>>>>>>
>>>>>>>>
>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>>>
>>>>>>> [4]
>>>>>>>> [4]
>>>>>>>> [2] mailto:openstack at lists.openstack.org [4] [5] [5]
>>>>>>>> [3]
>>>>>>>>
>>>>>>>>
>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>>>
>>>>>>> [5]
>>>>>>>> [6]
>>>>>>>> [6]
>>>>>>>> [4] mailto:giorgis at acmac.uoc.gr [6] [7] [7]
>>>>>>>>
>>>>>>>
>>>>>> Links:
>>>>>> ------
>>>>>> [1]
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>>> [7]
>>>>>> [8]
>>>>>> [2] mailto:openstack at lists.openstack.org [8] [9]
>>>>>> [3]
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>>> [9]
>>>>>> [10]
>>>>>> [4]
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>>> [10]
>>>>>> [11]
>>>>>> [5] mailto:openstack at lists.openstack.org [11] [12]
>>>>>> [6]
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>>> [12]
>>>>>> [13]
>>>>>> [7] mailto:giorgis at acmac.uoc.gr [13] [14]
>>>>>> [8] mailto:giorgis at acmac.uoc.gr [14] [15]
>>>>>>
>>>>>
>>>> Links:
>>>> ------
>>>> [1] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>> [15]
>>>> [2] mailto:openstack at lists.openstack.org [16]
>>>> [3] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>> [17]
>>>> [4] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>> [18]
>>>> [5] mailto:openstack at lists.openstack.org [19]
>>>> [6] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>> [20]
>>>> [7] mailto:giorgis at acmac.uoc.gr [21]
>>>> [8] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>> [22]
>>>> [9] mailto:openstack at lists.openstack.org [23]
>>>> [10]
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack [24]
>>>> [11]
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack [25]
>>>> [12] mailto:openstack at lists.openstack.org [26]
>>>> [13]
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack [27]
>>>> [14] mailto:giorgis at acmac.uoc.gr [28]
>>>> [15] mailto:giorgis at acmac.uoc.gr [29]
>>>> [16] mailto:giorgis at acmac.uoc.gr [30]
>>>>
>>>
>>
>> Links:
>> ------
>> [1] mailto:openstack at lists.openstack.org
>> [2] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> [3] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> [4] mailto:openstack at lists.openstack.org
>> [5] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> [6] mailto:giorgis at acmac.uoc.gr
>> [7] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> [8] mailto:openstack at lists.openstack.org
>> [9] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> [10] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> [11] mailto:openstack at lists.openstack.org
>> [12] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> [13] mailto:giorgis at acmac.uoc.gr
>> [14] mailto:giorgis at acmac.uoc.gr
>> [15] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> [16] mailto:openstack at lists.openstack.org
>> [17] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> [18] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> [19] mailto:openstack at lists.openstack.org
>> [20] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> [21] mailto:giorgis at acmac.uoc.gr
>> [22] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> [23] mailto:openstack at lists.openstack.org
>> [24] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> [25] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> [26] mailto:openstack at lists.openstack.org
>> [27] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> [28] mailto:giorgis at acmac.uoc.gr
>> [29] mailto:giorgis at acmac.uoc.gr
>> [30] mailto:giorgis at acmac.uoc.gr
>> [31] mailto:giorgis at acmac.uoc.gr
>>
>
> --
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20141216/64e93ed4/attachment.html>
More information about the Openstack
mailing list