[Openstack] Very slow connectivity from within tenant network - GRE

Geraint Jones geraint at koding.com
Wed Oct 23 00:48:00 UTC 2013


I have just had to tweak our grizzly network node the biggest impacts were
seem from doing changing root wrap to only use sudo – not the python wrapper
(its super slow) and upgrading openvswitch to 1.11

This smoke ping shows the latency to one of our instances from europe before
and after the openvswitch upgrade : http://d.pr/i/36v0

And this graph shows the load avg on our network node, the first drop is
from disabling root wrap the second is after the OVS upgrade :
http://d.pr/i/xhFc

I would suggest you do the same, and just make sure all MTU’s are correct.

-- 
Geraint Jones
Director of Systems & Infrastructure
Koding 
https://koding.com
geraint at koding.com
M (NZ) +64 22 123 4626
M (US) +1 415 316 8027

From:  Martinx - ジェームズ <thiagocmartinsc at gmail.com>
Date:  Tuesday, 22 October 2013 9:00 am
To:  Rick Jones <rick.jones2 at hp.com>
Cc:  "openstack at lists.openstack.org" <openstack at lists.openstack.org>
Subject:  Re: [Openstack] Very slow connectivity from within tenant network
- GRE

Hi Rick!

Back with Grizzly, I faced that problem and I was able to detect it, at the
Network Node with tcpdump and fix it by running "ip link set mtu 1454 dev
eth0" within the Instance.

Not this time... This is another problem...   ;-/


On 22 October 2013 13:25, Rick Jones <rick.jones2 at hp.com> wrote:
> On 10/22/2013 01:32 AM, Martinx - ジェームズ wrote:
>> Stackers,
>> 
>> I'm trying to put my Havana into production and I'm facing a very
>> strange problem.
>> 
>> The Internet connectivity from tenant's subnet is very, very slow. It is
>> useless in fact... I can not even use "apt-get update" from a Instance.
>> 
>> The following command works (apt update from the tenant namespace):
>> 
>> ---
>> root at net-node-1:~# ip netns exec qrouter-XXXXXXXXX aptitude update
>> ---
>> 
>> But not from the tenant subnet...
>> 
>> I'm following this topology:
>> 
>> http://docs.openstack.org/trunk/install-guide/install/apt/content/section_use
>> -cases-tenant-router.html
>> <http://docs.openstack.org/trunk/install-guide/install/apt/content/section_us
>> e-cases-tenant-router.html>
>> 
>> Already tried to change MTUs (via DHCP agent)... Nothing had fixed this
>> weird issue.
>> 
>> Any thoughts?!
>> 
>> Right now, my "aptitude safe-upgrade" will take 2 days to download
>> 60MB... During this network outages, even the SSH session stops
>> responding for a few seconds...
>> 
>> Everything else seems to be working as expected, as for example, DHCP,
>> Floating IPs, Security Groups...
>> 
>> Sometimes, even the first ssh connection to the Instance Floating IP,
>> have a lag.
> 
> It is but a guess, but I wonder if, even with changing MTUs (to what values?)
> you may still be experiencing a PathMTU+ICMP blackhole problem accessing nodes
> on the Internet.  Can you access something that is a bit "closer" but still
> outside your stack so you have a shot at looking at netstat statistics on the
> sender and/or get packet traces on the sender?
> 
> You could still try taking packet traces at the instance or perhaps the
> namespace and try to discern packet losses at the receiving side, though it
> can be a bit more difficult.
> 
> rick jones
> 

_______________________________________________ Mailing list:
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to     :
openstack at lists.openstack.org Unsubscribe :
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20131022/0eff67a2/attachment.html>


More information about the Openstack mailing list