<div dir="ltr"><div><div>Hi pedro,<br><br></div>This thread might interest you :<br><br><a href="http://lists.openstack.org/pipermail/openstack-dev/2015-January/054953.html">http://lists.openstack.org/pipermail/openstack-dev/2015-January/054953.html</a><br><br></div>Mathieu<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jan 23, 2015 at 12:07 PM, Pedro Sousa <span dir="ltr"><<a href="mailto:pgsousa@gmail.com" target="_blank">pgsousa@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Slawek,<div><br></div><div>I've tried with 8950/9000 but I had problems communicating with external hosts from the VM.</div><div><br></div><div>Regards,</div><div>Pedro Sousa</div><div><br></div><div><br></div><div><br></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jan 22, 2015 at 9:36 PM, Sławek Kapłoński <span dir="ltr"><<a href="mailto:slawek@kaplonski.pl" target="_blank">slawek@kaplonski.pl</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">As I wrote earlier, for me it is best to have 9000 on hosts and 8950 on instances. Then I have full speed between instances. With lower mtu on instances I have about 2-2.5 Gbps and I saw that vhost-net process on host is using 100 of 1 cpu core. I'm using libvirt with kvm - maybe You are using something else and it will be different on Your hosts.<br>
<br>
Slawek Kaplonski<br>
<br>
<br>
W dniu 22.01.2015 o 20:45, Pedro Sousa pisze:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Slawek,<br>
<br><span>
I've tried several options but that one that seems to work better is MTU<br>
1450 on VM and MTU 1600 on the host. With MTU 1400 on the VM I would get<br>
freezes and timeouts.<br>
<br>
Still I get about 2.2Gbit/Sec while in the host I get 9 Gbit/Sec, do you<br>
think is normal?<br>
<br>
Thanks,<br>
Pedro Sousa<br>
<br>
<br>
<br>
<br>
On Thu, Jan 22, 2015 at 7:32 PM, Sławek Kapłoński <<a href="mailto:slawek@kaplonski.pl" target="_blank">slawek@kaplonski.pl</a><br></span><span>
<mailto:<a href="mailto:slawek@kaplonski.pl" target="_blank">slawek@kaplonski.pl</a>>> wrote:<br>
<br>
Hello,<br>
<br>
In dnsmasq file in neutron will be ok. It will then force option 26<br>
on vm.<br>
You can also manually change it on vms to tests.<br>
<br>
Slawek Kaplonski<br>
<br>
W dniu 22.01.2015 o 17:06, Pedro Sousa pisze:<br>
<br>
Hi Slawek,<br>
<br>
I'll test this, did you change the mtu on dnsmasq file in<br>
/etc/neutron/?<br>
Or do you need to change on other places too?<br>
<br>
Thanks,<br>
Pedro Sousa<br>
<br>
On Wed, Jan 21, 2015 at 4:26 PM, Sławek Kapłoński<br>
<<a href="mailto:slawek@kaplonski.pl" target="_blank">slawek@kaplonski.pl</a> <mailto:<a href="mailto:slawek@kaplonski.pl" target="_blank">slawek@kaplonski.pl</a>><br></span><div><div>
<mailto:<a href="mailto:slawek@kaplonski.pl" target="_blank">slawek@kaplonski.pl</a> <mailto:<a href="mailto:slawek@kaplonski.pl" target="_blank">slawek@kaplonski.pl</a>>>> wrote:<br>
<br>
I have similar and I also got something like 2-2,5Gbps<br>
between vms.<br>
When I<br>
change it to 8950 on vms (so in neutron conf) (50 less then on<br>
hosts) then it<br>
is much better.<br>
You can check that probably when You make test between vms<br>
on host<br>
there is<br>
process called "vhost-net" (or something like that) and it<br>
uses 100%<br>
of one cpu<br>
core and that is imho bottleneck<br>
<br>
Slawek Kaplonski<br>
<br>
On Wed, Jan 21, 2015 at 04:12:02PM +0000, Pedro Sousa wrote:<br>
> Hi Slawek,<br>
><br>
> I have dhcp-option-force=26,1400 in neutron-dnsmasq.conf and<br>
MTU=9000 on<br>
> network-interfaces in the operating system.<br>
><br>
> Do I need to change somewhere else?<br>
><br>
> Thanks,<br>
> Pedro Sousa<br>
><br>
> On Wed, Jan 21, 2015 at 4:07 PM, Sławek Kapłoński<br>
<<a href="mailto:slawek@kaplonski.pl" target="_blank">slawek@kaplonski.pl</a> <mailto:<a href="mailto:slawek@kaplonski.pl" target="_blank">slawek@kaplonski.pl</a>><br></div></div>
<mailto:<a href="mailto:slawek@kaplonski.pl" target="_blank">slawek@kaplonski.pl</a> <mailto:<a href="mailto:slawek@kaplonski.pl" target="_blank">slawek@kaplonski.pl</a>>>><span><br>
<br>
> wrote:<br>
><br>
> > Hello,<br>
> ><br>
> > Try to set bigger jumbo framse on hosts and vms. For<br>
example on<br>
hosts You<br>
> > can<br>
> > set 9000 and then 8950 and check then. It helps me<br>
with similar<br>
problem.<br>
> ><br>
> > Slawek Kaplonski<br>
> ><br>
> > On Wed, Jan 21, 2015 at 03:22:50PM +0000, Pedro Sousa<br>
wrote:<br>
> > > Hi all,<br>
> > ><br>
> > > is there a way to improve network performance on my<br>
instances<br>
with<br>
> > VXLAN? I<br>
> > > changed the MTU on physical interfaces to 1600, still<br>
performance it's<br>
> > > lower than in baremetal hosts:<br>
> > ><br>
> > > *On Instance:*<br>
> > ><br></span>
> > > [root@vms6-149a71e8-1f2a-4d6e-<u></u>__bba4-e70dfa42b289<br>
~]# iperf3 -s<br>
> > ><br>
------------------------------<u></u>__----------------------------<u></u>-<br>
> > > Server listening on 5201<br>
> > ><br>
------------------------------<u></u>__----------------------------<u></u>-<div><div><br>
> > > Accepted connection from 10.0.66.35, port 42900<br>
> > > [ 5] local 10.0.66.38 port 5201 connected to<br>
10.0.66.35 port<br>
42901<br>
> > > [ ID] Interval Transfer Bandwidth<br>
> > > [ 5] 0.00-1.00 sec 189 MBytes 1.59 Gbits/sec<br>
> > > [ 5] 1.00-2.00 sec 245 MBytes 2.06 Gbits/sec<br>
> > > [ 5] 2.00-3.00 sec 213 MBytes 1.78 Gbits/sec<br>
> > > [ 5] 3.00-4.00 sec 227 MBytes 1.91 Gbits/sec<br>
> > > [ 5] 4.00-5.00 sec 235 MBytes 1.97 Gbits/sec<br>
> > > [ 5] 5.00-6.00 sec 235 MBytes 1.97 Gbits/sec<br>
> > > [ 5] 6.00-7.00 sec 234 MBytes 1.96 Gbits/sec<br>
> > > [ 5] 7.00-8.00 sec 235 MBytes 1.97 Gbits/sec<br>
> > > [ 5] 8.00-9.00 sec 244 MBytes 2.05 Gbits/sec<br>
> > > [ 5] 9.00-10.00 sec 234 MBytes 1.97 Gbits/sec<br>
> > > [ 5] 10.00-10.04 sec 9.30 MBytes 1.97 Gbits/sec<br>
> > > - - - - - - - - - - - - - - - - - - - - - - - - -<br>
> > > [ ID] Interval Transfer Bandwidth<br>
Retr<br>
> > > [ 5] 0.00-10.04 sec 2.25 GBytes 1.92<br>
Gbits/sec 43<br>
> > sender<br>
> > > [ 5] 0.00-10.04 sec 2.25 GBytes 1.92 Gbits/sec<br>
> > > receiver<br>
> > ><br>
> > ><br>
> > > *On baremetal:*<br>
> > > iperf3 -s<br>
> > > warning: this system does not seem to support IPv6 -<br>
trying IPv4<br>
> > ><br></div></div>
------------------------------<u></u>__----------------------------<u></u>-<br>
> > > Server listening on 5201<br>
> > ><br>
------------------------------<u></u>__----------------------------<u></u>-<div><div><br>
> > > Accepted connection from 172.16.21.4, port 51408<br>
> > > [ 5] local 172.16.21.5 port 5201 connected to<br>
172.16.21.4<br>
port 51409<br>
> > > [ ID] Interval Transfer Bandwidth<br>
> > > [ 5] 0.00-1.00 sec 1.02 GBytes 8.76 Gbits/sec<br>
> > > [ 5] 1.00-2.00 sec 1.07 GBytes 9.23 Gbits/sec<br>
> > > [ 5] 2.00-3.00 sec 1.08 GBytes 9.29 Gbits/sec<br>
> > > [ 5] 3.00-4.00 sec 1.08 GBytes 9.27 Gbits/sec<br>
> > > [ 5] 4.00-5.00 sec 1.08 GBytes 9.27 Gbits/sec<br>
> > > [ 5] 5.00-6.00 sec 1.08 GBytes 9.28 Gbits/sec<br>
> > > [ 5] 6.00-7.00 sec 1.08 GBytes 9.28 Gbits/sec<br>
> > > [ 5] 7.00-8.00 sec 1.08 GBytes 9.29 Gbits/sec<br>
> > > [ 5] 8.00-9.00 sec 1.08 GBytes 9.28 Gbits/sec<br>
> > > [ 5] 9.00-10.00 sec 1.08 GBytes 9.29 Gbits/sec<br>
> > > [ 5] 10.00-10.04 sec 42.8 MBytes 9.31 Gbits/sec<br>
> > > - - - - - - - - - - - - - - - - - - - - - - - - -<br>
> > > [ ID] Interval Transfer Bandwidth<br>
Retr<br>
> > > [ 5] 0.00-10.04 sec 10.8 GBytes 9.23<br>
Gbits/sec 95<br>
> > sender<br>
> > > [ 5] 0.00-10.04 sec 10.8 GBytes 9.22 Gbits/sec<br>
> > > receiver<br>
> > ><br>
> > ><br>
> > > Thanks,<br>
> > > Pedro Sousa<br>
> ><br></div></div>
> > > ______________________________<u></u>___________________<br>
> > > OpenStack-operators mailing list<br>
> > > OpenStack-operators@lists.__<a href="http://openstack.org" target="_blank">op<u></u>enstack.org</a><br>
<mailto:<a href="mailto:OpenStack-operators@lists.openstack.org" target="_blank">OpenStack-operators@<u></u>lists.openstack.org</a>><br>
<mailto:<a href="mailto:OpenStack-operators@" target="_blank">OpenStack-operators@</a>__<a href="http://lists.openstack.org" target="_blank"><u></u>lists.openstack.org</a><br>
<mailto:<a href="mailto:OpenStack-operators@lists.openstack.org" target="_blank">OpenStack-operators@<u></u>lists.openstack.org</a>>><br>
> > ><br>
<a href="http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-operators" target="_blank">http://lists.openstack.org/__<u></u>cgi-bin/mailman/listinfo/__<u></u>openstack-operators</a><span><br>
<<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack-operators</a>><br>
> ><br>
> ><br>
<br>
<br>
<br>
--<br>
Pozdrawiam<br>
Sławek Kapłonski<br></span>
<a href="mailto:slawek@kaplonski.pl" target="_blank">slawek@kaplonski.pl</a> <mailto:<a href="mailto:slawek@kaplonski.pl" target="_blank">slawek@kaplonski.pl</a>><br>
<br>
<br>
</blockquote><div><div>
<br>
-- <br>
Pozdrawiam<br>
Sławek Kapłonski<br>
<a href="mailto:slawek@kaplonski.pl" target="_blank">slawek@kaplonski.pl</a><br>
</div></div></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
OpenStack-operators mailing list<br>
<a href="mailto:OpenStack-operators@lists.openstack.org">OpenStack-operators@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
<br></blockquote></div><br></div>