<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"Formátovaný v HTML Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
span.FormtovanvHTMLChar
{mso-style-name:"Formátovaný v HTML Char";
mso-style-priority:99;
mso-style-link:"Formátovaný v HTML";
font-family:Consolas;}
span.StylE-mailovZprvy19
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
p.HTMLPreformatted, li.HTMLPreformatted, div.HTMLPreformatted
{mso-style-name:"HTML Preformatted";
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Courier New";}
span.StylE-mailovZprvy22
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=CS link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='color:#1F497D'>Sorry to shatter you expectations, but those numbers are perfectly OK.<o:p></o:p></span></p><p class=MsoNormal><span style='color:#1F497D'>I was testing on HPE DL380 gen9 with </span><a href="https://www.cpubenchmark.net/cpu.php?cpu=Intel+Xeon+E5-2630+v3+%40+2.40GHz&id=2386&cpuCount=2" target="_blank">Intel Xeon E5-2630v3</a> and I got these speeds between two KVM VMs on the same host using netperf<o:p></o:p></p><p class=MsoNormal><span style='color:#1F497D'>28186 Mb/s with linux bridge<o:p></o:p></span></p><p class=MsoNormal><span style='color:#1F497D'>18552 Mb/s with OpenVSwitch with the full Neutron setup with iptables.<o:p></o:p></span></p><p class=MsoNormal><span style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='color:#1F497D'>How much would you like to achieve? I got 38686 on dev lo on the physical server and 47894 on a VM. You could turn to OVS with DPDK as the data path, bu I doubt it will do much. SR-IOV might, but I never tried any of this. I’m satisfied with the speed for my purposes.<o:p></o:p></span></p><p class=MsoNormal><span style='color:#1F497D'>Tomas from Homeatcloud<o:p></o:p></span></p><p class=MsoNormal><span style='color:#1F497D'><o:p> </o:p></span></p><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Manuel Sopena Ballesteros [mailto:manuel.sb@garvan.org.au] <br><b>Sent:</b> Tuesday, April 18, 2017 9:11 AM<br><b>To:</b> openstack@lists.openstack.org<br><b>Subject:</b> [Openstack] poor bandwidth across instances running on same host<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><span lang=EN-US>Hi all,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>I created 2 instances on the same compute node and tested the bandwidth between them, surprisingly iperf tells me I got 16.1Gbits/sec only. Then I changed the firewall from hybrid iptables to ovs, the bandwidth improved a little bit to 17.5Gbits/sec but still far from expected. <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p><p class=MsoNormal><b><u><span lang=EN-US>Ml2_config.ini config file</span></u></b><span lang=EN-US><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p><pre><span lang=EN-US style='color:black'>[root@nova-compute ~]# docker exec -t neutron_openvswitch_agent vi /var/lib/kolla/config_files/ml2_config.ini</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>network_vlan_ranges =</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> </span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>[ml2_type_flat]</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>flat_networks = physnet1</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> </span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>[ml2_type_vxlan]</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>vni_ranges = 1:1000</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>vxlan_group = 239.1.1.1</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> </span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>[securitygroup]</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>firewall_driver = openvswitch</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>#firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> </span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>[agent]</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>tunnel_types = vxlan</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>l2_population = true</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>arp_responder = true</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> </span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>[ovs]</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>bridge_mappings = physnet1:br-ex</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>ovsdb_connection = tcp:129.94.72.54:6640</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>local_ip = 10.1.0.12</span><span lang=EN-US><o:p></o:p></span></pre><p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p><p class=MsoNormal><b><u><span lang=EN-US>ovs config</span></u></b><span lang=EN-US><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p><pre><span lang=EN-US style='color:black'>[root@nova-compute ~]# docker exec openvswitch_vswitchd ovs-vsctl show</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>306d62c4-8e35-45e0-838e-53ebe81f1d06</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Bridge br-ex</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Controller "tcp:127.0.0.1:6633"</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> is_connected: true</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> fail_mode: secure</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Port "eno50336512"</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Interface "eno50336512"</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Port phy-br-ex</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Interface phy-br-ex</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> type: patch</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> options: {peer=int-br-ex}</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Port br-ex</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Interface br-ex</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> type: internal</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Bridge br-tun</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Controller "tcp:127.0.0.1:6633"</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> is_connected: true</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> fail_mode: secure</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Port patch-int</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Interface patch-int</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> type: patch</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> options: {peer=patch-tun}</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Port "vxlan-0a01000b"</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Interface "vxlan-0a01000b"</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> type: vxlan</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> options: {df_default="true", in_key=flow, local_ip="10.1.0.12", out_key=flow, remote_ip="10.1.0.11"}</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Port br-tun</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Interface br-tun</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> type: internal</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Bridge br-int</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Controller "tcp:127.0.0.1:6633"</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> is_connected: true</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> fail_mode: secure</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Port int-br-ex</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Interface int-br-ex</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> type: patch</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> options: {peer=phy-br-ex}</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Port "tapa26ee521-3b"</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> tag: 2</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Interface "tapa26ee521-3b"</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Port patch-tun</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Interface patch-tun</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> type: patch</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> options: {peer=patch-int}</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Port br-int</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Interface br-int</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> type: internal</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Port "tap1f76851b-ea"</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> tag: 2</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> Interface "tap1f76851b-ea"</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> </span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> </span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'> </span><span lang=EN-US><o:p></o:p></span></pre><p class=MsoNormal><b><u><span lang=EN-US>Iperf results</span></u></b><span lang=EN-US><o:p></o:p></span></p><pre><span lang=EN-US style='color:black'> </span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>[centos@centos7 ~]$ iperf -c 192.168.1.105</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>------------------------------------------------------------</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>Client connecting to 192.168.1.105, TCP port 5001</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>TCP window size: 45.0 KByte (default)</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>------------------------------------------------------------</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>[ 3] local 192.168.1.101 port 48522 connected with 192.168.1.105 port 5001</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>[ ID] Interval Transfer Bandwidth</span><span lang=EN-US><o:p></o:p></span></pre><pre><span lang=EN-US style='color:black'>[ 3] 0.0-10.0 sec 20.3 GBytes 17.5 Gbits/sec</span><span lang=EN-US><o:p></o:p></span></pre><p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p><p class=MsoNormal><b><u><span lang=EN-US>Ovs info</span></u></b><span lang=EN-US><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Courier New";color:black'>[root@nova-compute ~]# docker exec openvswitch_vswitchd modinfo openvswitch</span><span lang=EN-US><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Courier New";color:black'>filename: /lib/modules/3.10.0-514.el7.x86_64/kernel/net/openvswitch/openvswitch.ko</span><span lang=EN-US><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Courier New";color:black'>license: GPL</span><span lang=EN-US><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Courier New";color:black'>description: Open vSwitch switching datapath</span><span lang=EN-US><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Courier New";color:black'>rhelversion: 7.3</span><span lang=EN-US><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Courier New";color:black'>srcversion: B31AE95554C9D9A0067F935</span><span lang=EN-US><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Courier New";color:black'>depends: nf_conntrack,nf_nat,libcrc32c,nf_nat_ipv6,nf_nat_ipv4,nf_defrag_ipv6</span><span lang=EN-US><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Courier New";color:black'>intree: Y</span><span lang=EN-US><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Courier New";color:black'>vermagic: 3.10.0-514.el7.x86_64 SMP mod_unload modversions</span><span lang=EN-US><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Courier New";color:black'>signer: CentOS Linux kernel signing key</span><span lang=EN-US><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Courier New";color:black'>sig_key: D4:88:63:A7:C1:6F:CC:27:41:23:E6:29:8F:74:F0:57:AF:19:FC:54</span><span lang=EN-US><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Courier New";color:black'>sig_hashalgo: sha256</span><span lang=EN-US><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>As far as I know the communication is VM </span><span lang=EN-US style='font-family:Wingdings'>ß</span><span lang=EN-US>OVS</span><span lang=EN-US style='font-family:Wingdings'>à</span><span lang=EN-US>VM and the linux bridge is not involved.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>What could be throttling the network traffic and what can I do to improve performance?<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>Thank you very much<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p><p class=MsoNormal><b><span lang=EN-US style='color:#333333;background:white'>Manuel Sopena Ballesteros </span></b><span lang=EN-US style='color:#333333;background:white'>|<b> </b>Big data Engineer</span><span lang=EN-US style='color:#333333'><br><b><span style='background:white'>Garvan Institute of Medical Research </span></b><br><span style='background:white'>The Kinghorn Cancer Centre,<b> </b>370 Victoria Street, Darlinghurst, NSW 2010</span><br><b><span style='background:white'>T:</span></b><span style='background:white'> + 61 (0)2 9355 5760 | <b>F:</b> +61 (0)2 9295 8507 | <b>E:</b> </span></span><span lang=EN-US style='color:black'><a href="mailto:manuel.sb@garvan.org.au">manuel.sb@garvan.org.au</a></span><span lang=EN-US><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p><div><p class=MsoNormal><span lang=EN-US style='font-size:8.0pt;font-family:"Arial","sans-serif"'>NOTICE<o:p></o:p></span></p></div><div><p class=MsoNormal><span lang=EN-US style='font-size:8.0pt;font-family:"Arial","sans-serif"'>Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain legally privileged/confidential/copyright information. If you are not the intended recipient, you should not read, use, disclose, copy or distribute this communication. If you have received this message in error please notify us at once by return email and then delete both messages. We accept no liability for the distribution of viruses or similar in electronic communications. This notice should not be removed. <o:p></o:p></span></p></div></div></body></html>