[Openstack] Directional network performance issues with Neutron + OpenvSwitch

Aaron Rosen arosen at nicira.com
Thu Oct 24 00:40:44 UTC 2013


On Mon, Oct 21, 2013 at 11:52 PM, Martinx - ジェームズ <thiagocmartinsc at gmail.com
> wrote:

> James,
>
> I think I'm hitting this problem.
>
> I'm using "Per-Tenant Routers with Private Networks", GRE tunnels and
> L3+DHCP Network Node.
>
> The connectivity from behind my Instances is very slow. It takes an
> eternity to finish "apt-get update".
>


I'm curious if you can do the following tests to help pinpoint the bottle
neck:

Run iperf or netperf between:
two instances on the same hypervisor - this will determine if it's a
virtualization driver issue if the performance is bad.
two instances on different hypervisors.
one instance to the namespace of the l3 agent.






>
> If I run "apt-get update" from within tenant's Namespace, it goes fine.
>
> If I enable "ovs_use_veth", Metadata (and/or DHCP) stops working and I and
> unable to start new Ubuntu Instances and login into them... Look:
>
> --
> cloud-init start running: Tue, 22 Oct 2013 05:57:39 +0000. up 4.01 seconds
> 2013-10-22 06:01:42,989 - util.py[WARNING]: '
> http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [3/120s]:
> url error [[Errno 113] No route to host]
> 2013-10-22 06:01:45,988 - util.py[WARNING]: '
> http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [6/120s]:
> url error [[Errno 113] No route to host]
> --
>


Do you see anything interesting in the neutron-metadata-agent log? Or it
looks like your instance doesn't have a route to the default gw?


>
> Is this problem still around?!
>
> Should I stay away from GRE tunnels when with Havana + Ubuntu 12.04.3?
>
> Is it possible to re-enable Metadata when ovs_use_veth = true ?
>
> Thanks!
> Thiago
>
>
> On 3 October 2013 06:27, James Page <james.page at ubuntu.com> wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA256
>>
>> On 02/10/13 22:49, James Page wrote:
>> >> sudo ip netns exec qrouter-d3baf1b1-55ee-42cb-a3f6-9629288e3221
>> >>> traceroute -n 10.5.0.2 -p 44444 --mtu traceroute to 10.5.0.2
>> >>> (10.5.0.2), 30 hops max, 65000 byte packets 1  10.5.0.2  0.950
>> >>> ms F=1500  0.598 ms  0.566 ms
>> >>>
>> >>> The PMTU from the l3 gateway to the instance looks OK to me.
>> > I spent a bit more time debugging this; performance from within
>> > the router netns on the L3 gateway node looks good in both
>> > directions when accessing via the tenant network (10.5.0.2) over
>> > the qr-XXXXX interface, but when accessing through the external
>> > network from within the netns I see the same performance choke
>> > upstream into the tenant network.
>> >
>> > Which would indicate that my problem lies somewhere around the
>> > qg-XXXXX interface in the router netns - just trying to figure out
>> > exactly what - maybe iptables is doing something wonky?
>>
>> OK - I found a fix but I'm not sure why this makes a difference;
>> neither my l3-agent or dhcp-agent configuration had 'ovs_use_veth =
>> True'; I switched this on, clearing everything down, rebooted and now
>> I seem symmetric good performance across all neutron routers.
>>
>> This would point to some sort of underlying bug when ovs_use_veth = False.
>>
>>
>> - --
>> James Page
>> Ubuntu and Debian Developer
>> james.page at ubuntu.com
>> jamespage at debian.org
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1.4.14 (GNU/Linux)
>> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>>
>> iQIcBAEBCAAGBQJSTTh6AAoJEL/srsug59jDmpEP/jaB5/yn9+Xm12XrVu0Q3IV5
>> fLGOuBboUgykVVsfkWccI/oygNlBaXIcDuak/E4jxPcoRhLAdY1zpX8MQ8wSsGKd
>> CjSeuW8xxnXubdfzmsCKSs3FCIBhDkSYzyiJd/raLvCfflyy8Cl7KN2x22mGHJ6z
>> qZ9APcYfm9qCVbEssA3BHcUL+st1iqMJ0YhVZBk03+QEXaWu3FFbjpjwx3X1ZvV5
>> Vbac7enqy7Lr4DSAIJVldeVuRURfv3YE3iJZTIXjaoUCCVTQLm5OmP9TrwBNHLsA
>> 7W+LceQri+Vh0s4dHPKx5MiHsV3RCydcXkSQFYhx7390CXypMQ6WwXEY/a8Egssg
>> SuxXByHwEcQFa+9sCwPQ+RXCmC0O6kUi8EPmwadjI5Gc1LoKw5Wov/SEen86fDUW
>> P9pRXonseYyWN9I4MT4aG1ez8Dqq/SiZyWBHtcITxKI2smD92G9CwWGo4L9oGqJJ
>> UcHRwQaTHgzy3yETPO25hjax8ZWZGNccHBixMCZKegr9p2dhR+7qF8G7mRtRQLxL
>> 0fgOAExn/SX59ZT4RaYi9fI6Gng13RtSyI87CJC/50vfTmqoraUUK1aoSjIY4Dt+
>> DYEMMLp205uLEj2IyaNTzykR0yh3t6dvfpCCcRA/xPT9slfa0a7P8LafyiWa4/5c
>> jkJM4Y1BUV+2L5Rrf3sc
>> =4lO4
>> -----END PGP SIGNATURE-----
>>
>> _______________________________________________
>> Mailing list:
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> Post to     : openstack at lists.openstack.org
>> Unsubscribe :
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>
>
>
> _______________________________________________
> Mailing list:
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to     : openstack at lists.openstack.org
> Unsubscribe :
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20131023/5128fec2/attachment.html>


More information about the Openstack mailing list