Open Stack

Sure, always... For example, from within a tenant instance:

------
ubuntu at instance-1:~$ ip route
default via 192.168.210.1 dev eth0  metric 100
192.168.210.0/24 dev eth0  proto kernel  scope link  src 192.168.210.2

# 192.168.210.1 resides within the Tenant Namespace at the Network Node,
ping ok:
ubuntu at instance-1-1:~$ ping -c 1 192.168.210.1


PING 192.168.210.1 (192.168.210.1) 56(84) bytes of data.
64 bytes from 192.168.210.1: icmp_req=1 ttl=64 time=0.898 ms

--- 192.168.210.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.898/0.898/0.898/0.000 ms

# Internet...
ubuntu at instance-1:~$ ping -c 1 8.8.8.8


PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_req=1 ttl=48 time=133 ms

--- 8.8.8.8 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 133.232/133.232/133.232/0.000 ms

# Very slow speed:
ubuntu at instance-1:~$ sudo aptitude update
Get: 1 http://nova.clouds.archive.ubuntu.com precise Release.gpg [198 B]
Get: 2 http://security.ubuntu.com precise-security Release.gpg [198 B]
Get: 3 http://nova.clouds.archive.ubuntu.com precise-updates Release.gpg
[198 B]
Get: 4 http://security.ubuntu.com precise-security Release [49.6 kB]
Get: 5 http://nova.clouds.archive.ubuntu.com precise-backports Release.gpg
[198 B]
Hit http://nova.clouds.archive.ubuntu.com precise Release
Get: 6 http://nova.clouds.archive.ubuntu.com precise-updates Release [49.6
kB]
81% [6 Release 44.5 kB/49.6 kB 90%] [4 Release 35.4 kB/49.6 kB 71%]


    *7,521
B/s 2s*
------

7,521 B/s ???

If I run "aptitude update" from within Tenant's Namespace, the "router"
with the IP 192.168.210.1 (tenant's gateway), it goes just fine. Look:

------
root at net-node-1:~# ip netns exec
qrouter-46cb8f7a-a3c5-4da7-ad69-4de63f7c34f1 ip r
default via 172.16.0.1 dev qg-50b615b7-c2
172.16.0.0/20 dev qg-50b615b7-c2  proto kernel  scope link  src 172.16.0.2
192.168.210.0/24 dev qr-a1376f61-05  proto kernel  scope link  src
192.168.210.1

# Normal speed
root at net-node-1:~# ip netns exec
qrouter-46cb8f7a-a3c5-4da7-ad69-4de63f7c34f1 aptitude update
Hit http://us.archive.ubuntu.com precise Release.gpg

Hit http://us.archive.ubuntu.com precise-updates Release.gpg
Get: 1 http://us.archive.ubuntu.com precise-backports Release.gpg [198 B]
Get: 2 http://ubuntu-cloud.archive.canonical.com precise-updates/havana
Release.gpg [543 B]
Get: 3 http://security.ubuntu.com precise-security Release.gpg [198 B]
....sniped....
------

No idea about what's happening...    ;-(

Also, the Compute Node have normal speed too, where the instance-1 is
running.

This Havana I'm trying to put into production, follow a well tested
topology that I already have up and running, very similar with this:
https://github.com/mseknibilel/OpenStack-Grizzly-Install-Guide/blob/OVS_MultiNode/OpenStack_Grizzly_Install_Guide.rst

I'll (double * double) check everything again. I like to test and debug...
    :-D

Best,
Thiago


On 23 October 2013 01:52, 止语 <menghuizhanguo at gmail.com> wrote:

> Have you ever try to check the route ?
> #route -n
>
>
> ------------------ Original ------------------
> *From: * "Martinx - ジェームズ";<thiagocmartinsc at gmail.com>;
> *Date: * Wed, Oct 23, 2013 11:34 AM
> *To: * "Geraint Jones"<geraint at koding.com>; **
> *Cc: * "openstack at lists.openstack.org"<openstack at lists.openstack.org>; **
> *Subject: * Re: [Openstack] Very slow connectivity from within tenant
> network -GRE
>
> WOW! Nice move!
>
> But, upgrading from `openvswitch 1.10.2-0ubuntu2~cloud0` to `1.11.0-1` did
> not solved my issue.
>
> Tenant Instances still have a very slow Internet connectivity.
>
> Thanks anyway! Nice to see your charts, pretty good improvement!
>
> Regards,
> Thiago
>
>
> On 22 October 2013 22:48, Geraint Jones <geraint at koding.com> wrote:
>
>> I have just had to tweak our grizzly network node the biggest impacts
>> were seem from doing changing root wrap to only use sudo – not the python
>> wrapper (its super slow) and upgrading openvswitch to 1.11
>>
>> This smoke ping shows the latency to one of our instances from europe
>> before and after the openvswitch upgrade : http://d.pr/i/36v0
>>
>> And this graph shows the load avg on our network node, the first drop is
>> from disabling root wrap the second is after the OVS upgrade :
>> http://d.pr/i/xhFc
>>
>> I would suggest you do the same, and just make sure all MTU’s are correct.
>>
>> --
>> Geraint Jones
>> Director of Systems & Infrastructure
>> Koding
>> https://koding.com
>> geraint at koding.com
>> M (NZ) +64 22 123 4626
>> M (US) +1 415 316 8027
>>
>> From: Martinx - ジェームズ <thiagocmartinsc at gmail.com>
>> Date: Tuesday, 22 October 2013 9:00 am
>> To: Rick Jones <rick.jones2 at hp.com>
>> Cc: "openstack at lists.openstack.org" <openstack at lists.openstack.org>
>> Subject: Re: [Openstack] Very slow connectivity from within tenant
>> network - GRE
>>
>> Hi Rick!
>>
>> Back with Grizzly, I faced that problem and I was able to detect it, at
>> the Network Node with tcpdump and fix it by running "ip link set mtu 1454
>> dev eth0" within the Instance.
>>
>> Not this time... This is another problem...   ;-/
>>
>>
>> On 22 October 2013 13:25, Rick Jones <rick.jones2 at hp.com> wrote:
>>
>>> On 10/22/2013 01:32 AM, Martinx - ジェームズ wrote:
>>>
>>>> Stackers,
>>>>
>>>> I'm trying to put my Havana into production and I'm facing a very
>>>> strange problem.
>>>>
>>>> The Internet connectivity from tenant's subnet is very, very slow. It is
>>>> useless in fact... I can not even use "apt-get update" from a Instance.
>>>>
>>>> The following command works (apt update from the tenant namespace):
>>>>
>>>> ---
>>>> root at net-node-1:~# ip netns exec qrouter-XXXXXXXXX aptitude update
>>>> ---
>>>>
>>>> But not from the tenant subnet...
>>>>
>>>> I'm following this topology:
>>>>
>>>> http://docs.openstack.org/**trunk/install-guide/install/**
>>>> apt/content/section_use-cases-**tenant-router.html<http://docs.openstack.org/trunk/install-guide/install/apt/content/section_use-cases-tenant-router.html>
>>>>
>>>> Already tried to change MTUs (via DHCP agent)... Nothing had fixed this
>>>> weird issue.
>>>>
>>>> Any thoughts?!
>>>>
>>>> Right now, my "aptitude safe-upgrade" will take 2 days to download
>>>> 60MB... During this network outages, even the SSH session stops
>>>> responding for a few seconds...
>>>>
>>>> Everything else seems to be working as expected, as for example, DHCP,
>>>> Floating IPs, Security Groups...
>>>>
>>>> Sometimes, even the first ssh connection to the Instance Floating IP,
>>>> have a lag.
>>>>
>>>
>>> It is but a guess, but I wonder if, even with changing MTUs (to what
>>> values?) you may still be experiencing a PathMTU+ICMP blackhole problem
>>> accessing nodes on the Internet.  Can you access something that is a bit
>>> "closer" but still outside your stack so you have a shot at looking at
>>> netstat statistics on the sender and/or get packet traces on the sender?
>>>
>>> You could still try taking packet traces at the instance or perhaps the
>>> namespace and try to discern packet losses at the receiving side, though it
>>> can be a bit more difficult.
>>>
>>> rick jones
>>>
>>>
>> _______________________________________________ Mailing list:
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to :
>> openstack at lists.openstack.org Unsubscribe :
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20131023/98961bab/attachment.html>