<div dir="ltr">Sure, always... For example, from within a tenant instance:<div><br></div><div><div>------<br></div><div>ubuntu@instance-1:~$ ip route</div><div>default via <a href="tel:192.168.210.1" value="+551921682101" target="_blank">192.168.210.1</a> dev eth0 metric 100 </div>
<div><a href="http://192.168.210.0/24" target="_blank">192.168.210.0/24</a> dev eth0 proto kernel scope link src <a href="tel:192.168.210.2" value="+551921682102" target="_blank">192.168.210.2</a> </div>
<div><br></div><div># <a href="tel:192.168.210.1" value="+551921682101" target="_blank">192.168.210.1</a> resides within the Tenant Namespace at the Network Node, ping ok:</div><div>ubuntu@instance-1-1:~$ ping -c <a href="tel:1%20192.168.210.1" value="+5511921682101" target="_blank">1 192.168.210.1</a> </div>
<div>PING <a href="tel:192.168.210.1" value="+551921682101" target="_blank">192.168.210.1</a> <a href="tel:%28192.168.210.1" value="+551921682101" target="_blank">(192.168.210.1</a>) 56(84) bytes of data.</div><div>64 bytes from <a href="http://192.168.210.1" target="_blank">192.168.210.1</a>: icmp_req=1 ttl=64 time=0.898 ms</div>
<div><br></div><div>--- <a href="tel:192.168.210.1" value="+551921682101" target="_blank">192.168.210.1</a> ping statistics ---</div>
<div>1 packets transmitted, 1 received, 0% packet loss, time 0ms</div><div>rtt min/avg/max/mdev = 0.898/0.898/0.898/0.000 ms</div><div><br></div><div># Internet...</div><div>ubuntu@instance-1:~$ ping -c 1 8.8.8.8 </div>
<div>PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.</div><div>64 bytes from <a href="http://8.8.8.8" target="_blank">8.8.8.8</a>: icmp_req=1 ttl=48 time=133 ms</div><div><br></div><div>--- 8.8.8.8 ping statistics ---</div>
<div>1 packets transmitted, 1 received, 0% packet loss, time 0ms</div>
<div>rtt min/avg/max/mdev = 133.232/133.232/133.232/0.000 ms</div><div><br></div><div># Very slow speed:</div><div>ubuntu@instance-1:~$ sudo aptitude update</div><div>Get: 1 <a href="http://nova.clouds.archive.ubuntu.com" target="_blank">http://nova.clouds.archive.ubuntu.com</a> precise Release.gpg [198 B]</div>
<div>Get: 2 <a href="http://security.ubuntu.com" target="_blank">http://security.ubuntu.com</a> precise-security Release.gpg [198 B]</div><div>Get: 3 <a href="http://nova.clouds.archive.ubuntu.com" target="_blank">http://nova.clouds.archive.ubuntu.com</a> precise-updates Release.gpg [198 B]</div>
<div>Get: 4 <a href="http://security.ubuntu.com" target="_blank">http://security.ubuntu.com</a> precise-security Release [49.6 kB]</div><div>Get: 5 <a href="http://nova.clouds.archive.ubuntu.com" target="_blank">http://nova.clouds.archive.ubuntu.com</a> precise-backports Release.gpg [198 B]</div>
<div>Hit <a href="http://nova.clouds.archive.ubuntu.com" target="_blank">http://nova.clouds.archive.ubuntu.com</a> precise Release</div><div>Get: 6 <a href="http://nova.clouds.archive.ubuntu.com" target="_blank">http://nova.clouds.archive.ubuntu.com</a> precise-updates Release [49.6 kB]</div>
<div>81% [6 Release 44.5 kB/49.6 kB 90%] [4 Release 35.4 kB/49.6 kB 71%] <b>7,521 B/s 2s</b></div>
</div><div>------</div><div><br></div><div>7,521 B/s ???<br></div><div><br></div><div>If I run "aptitude update" from within Tenant's Namespace, the "router" with the IP <a href="tel:192.168.210.1" value="+551921682101" target="_blank">192.168.210.1</a> (tenant's gateway), it goes just fine. Look:</div>
<div><br></div><div>------</div><div><div>root@net-node-1:~# ip netns exec qrouter-46cb8f7a-a3c5-4da7-ad69-4de63f7c34f1 ip r</div><div>default via 172.16.0.1 dev qg-50b615b7-c2 </div><div><a href="http://172.16.0.0/20" target="_blank">172.16.0.0/20</a> dev qg-50b615b7-c2 proto kernel scope link src 172.16.0.2 </div>
<div><a href="http://192.168.210.0/24" target="_blank">192.168.210.0/24</a> dev qr-a1376f61-05 proto kernel scope link src <a href="tel:192.168.210.1" value="+551921682101" target="_blank">192.168.210.1</a> </div><div>
<br></div><div># Normal speed</div><div>root@net-node-1:~# ip netns exec qrouter-46cb8f7a-a3c5-4da7-ad69-4de63f7c34f1 aptitude update</div>
<div>Hit <a href="http://us.archive.ubuntu.com" target="_blank">http://us.archive.ubuntu.com</a> precise Release.gpg </div><div>Hit <a href="http://us.archive.ubuntu.com" target="_blank">http://us.archive.ubuntu.com</a> precise-updates Release.gpg</div>
<div>Get: 1 <a href="http://us.archive.ubuntu.com" target="_blank">http://us.archive.ubuntu.com</a> precise-backports Release.gpg [198 B]</div><div>Get: 2 <a href="http://ubuntu-cloud.archive.canonical.com" target="_blank">http://ubuntu-cloud.archive.canonical.com</a> precise-updates/havana Release.gpg [543 B]</div>
<div>Get: 3 <a href="http://security.ubuntu.com" target="_blank">http://security.ubuntu.com</a> precise-security Release.gpg [198 B]</div></div><div>....sniped....</div><div>------</div><div><br></div><div>No idea about what's happening... ;-(<br>
</div><div><br></div><div>Also, the Compute Node have normal speed too, where the instance-1 is running.</div><div><br></div><div>This Havana I'm trying to put into production, follow a well tested topology that I already have up and running, very similar with this: <a href="https://github.com/mseknibilel/OpenStack-Grizzly-Install-Guide/blob/OVS_MultiNode/OpenStack_Grizzly_Install_Guide.rst" target="_blank">https://github.com/mseknibilel/OpenStack-Grizzly-Install-Guide/blob/OVS_MultiNode/OpenStack_Grizzly_Install_Guide.rst</a></div>
<div><br></div><div>I'll (double * double) check everything again. I like to test and debug... :-D</div><div><br></div><div>Best,</div><div>Thiago</div><div class="gmail_extra"><br><br><div class="gmail_quote">
On 23 October 2013 01:52, 止语 <span dir="ltr"><<a href="mailto:menghuizhanguo@gmail.com" target="_blank">menghuizhanguo@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Have you ever try to check the route ?<br>#route -n <br><div><div><br></div><div><br></div><div style="font-size:12px;font-family:Arial Narrow;padding:2px 0 2px 0">------------------ Original ------------------</div><div style="font-size:12px;background:#efefef;padding:8px">
<div><div><b>From: </b> "Martinx - ジェームズ";<<a href="mailto:thiagocmartinsc@gmail.com" target="_blank">thiagocmartinsc@gmail.com</a>>;</div></div><div><b>Date: </b> Wed, Oct 23, 2013 11:34 AM</div>
<div><b>To: </b> "Geraint Jones"<<a href="mailto:geraint@koding.com" target="_blank">geraint@koding.com</a>>; <u></u></div><div><div><div><b>Cc: </b> "<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a>"<<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a>>; <u></u></div>
<div><b>Subject: </b> Re: [Openstack] Very slow connectivity from within tenant network -GRE</div></div></div></div><div><div><div><br></div><div dir="ltr">WOW! Nice move!<div><br></div><div>But, upgrading from `openvswitch 1.10.2-0ubuntu2~cloud0` to `1.11.0-1` did not solved my issue. </div>
<div><br></div><div>Tenant Instances still have a very slow Internet connectivity.</div>
<div><br></div><div>Thanks anyway! Nice to see your charts, pretty good improvement!</div><div><br></div><div>Regards,</div><div>Thiago</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On 22 October 2013 22:48, Geraint Jones <span dir="ltr"><<a href="mailto:geraint@koding.com" target="_blank">geraint@koding.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word"><div><div><div>I have just had to tweak our grizzly network node the biggest impacts were seem from doing changing root wrap to only use sudo – not the python wrapper (its super slow) and upgrading openvswitch to 1.11</div>
<div><br></div><div>This smoke ping shows the latency to one of our instances from europe before and after the openvswitch upgrade : <a href="http://d.pr/i/36v0" target="_blank">http://d.pr/i/36v0</a></div><div><br></div>
<div>And this graph shows the load avg on our network node, the first drop is from disabling root wrap the second is after the OVS upgrade : <a href="http://d.pr/i/xhFc" target="_blank">http://d.pr/i/xhFc</a></div><div><br>
</div><div>I would suggest you do the same, and just make sure all MTU’s are correct.</div><div><br></div><div><div><div>-- </div><div>Geraint Jones</div></div><div>Director of Systems & Infrastructure</div><div>Koding </div>
<div><a href="https://koding.com" target="_blank">https://koding.com</a></div><div><a href="mailto:geraint@koding.com" target="_blank">geraint@koding.com</a></div><div>M (NZ) <a value="+64221234626">+64 22 123 4626</a></div>
<div>M (US) <a value="+14153168027">+1 415 316 8027</a></div></div></div></div><div><br></div><span><div style="border-right:medium none;padding-right:0in;padding-left:0in;padding-top:3pt;text-align:left;font-size:11pt;border-bottom:medium none;font-family:Calibri;border-top:#b5c4df 1pt solid;padding-bottom:0in;border-left:medium none">
<span style="font-weight:bold">From: </span> Martinx - ジェームズ <<a href="mailto:thiagocmartinsc@gmail.com" target="_blank">thiagocmartinsc@gmail.com</a>><br><span style="font-weight:bold">Date: </span> Tuesday, 22 October 2013 9:00 am<br>
<span style="font-weight:bold">To: </span> Rick Jones <<a href="mailto:rick.jones2@hp.com" target="_blank">rick.jones2@hp.com</a>><br><span style="font-weight:bold">Cc: </span> "<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a>" <<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a>><br>
<span style="font-weight:bold">Subject: </span> Re: [Openstack] Very slow connectivity from within tenant network - GRE<br></div><div><div><div><br></div><div dir="ltr">Hi Rick!<div><br></div><div>Back with Grizzly, I faced that problem and I was able to detect it, at the Network Node with tcpdump and fix it by running "ip link set mtu 1454 dev eth0" within the Instance.</div>
<div><br></div><div>Not this time... This is another problem... ;-/</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On 22 October 2013 13:25, Rick Jones <span dir="ltr"><<a href="mailto:rick.jones2@hp.com" target="_blank">rick.jones2@hp.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div>On 10/22/2013 01:32 AM, Martinx - ジェームズ wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Stackers,<br><br>
I'm trying to put my Havana into production and I'm facing a very<br>
strange problem.<br><br>
The Internet connectivity from tenant's subnet is very, very slow. It is<br>
useless in fact... I can not even use "apt-get update" from a Instance.<br><br>
The following command works (apt update from the tenant namespace):<br><br>
---<br>
root@net-node-1:~# ip netns exec qrouter-XXXXXXXXX aptitude update<br>
---<br><br>
But not from the tenant subnet...<br><br>
I'm following this topology:<br><br><a href="http://docs.openstack.org/trunk/install-guide/install/apt/content/section_use-cases-tenant-router.html" target="_blank">http://docs.openstack.org/<u></u>trunk/install-guide/install/<u></u>apt/content/section_use-cases-<u></u>tenant-router.html</a><br>
<br>
Already tried to change MTUs (via DHCP agent)... Nothing had fixed this<br>
weird issue.<br><br>
Any thoughts?!<br><br>
Right now, my "aptitude safe-upgrade" will take 2 days to download<br>
60MB... During this network outages, even the SSH session stops<br>
responding for a few seconds...<br><br>
Everything else seems to be working as expected, as for example, DHCP,<br>
Floating IPs, Security Groups...<br><br>
Sometimes, even the first ssh connection to the Instance Floating IP,<br>
have a lag.<br></blockquote><br></div></div>
It is but a guess, but I wonder if, even with changing MTUs (to what values?) you may still be experiencing a PathMTU+ICMP blackhole problem accessing nodes on the Internet. Can you access something that is a bit "closer" but still outside your stack so you have a shot at looking at netstat statistics on the sender and/or get packet traces on the sender?<br>
<br>
You could still try taking packet traces at the instance or perhaps the namespace and try to discern packet losses at the receiving side, though it can be a bit more difficult.<span><font color="#888888"><br><br>
rick jones<br><br></font></span></blockquote></div><br></div></div></div>
_______________________________________________
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a>
Post to : <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a>
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a>
</span></div>
</blockquote></div><br></div></div></div></div></blockquote></div><br></div></div>