Open Stack

We ran into an issue (change of default behavior) with GRE/VxLAN tunnel
fragmentation when we moved to the 3.13 kernel.  One workaround for this
issue is to set "options:df_default=false" on the ovs tunnels, see [1].
Fragmenting those packets is not ideal, but it would be interesting to
see if this is the problem you’re running into.

- Jack

[1] https://review.openstack.org/#/c/75281/9/neutron/agent/linux/ovs_lib.py

From: Martinx - ジェームズ [mailto:thiagocmartinsc at gmail.com]
Sent: Tuesday, May 20, 2014 12:03 PM
To: Jason Bishop
Cc: openstack at lists.openstack.org
Subject: Re: [Openstack] IceHouse Neutron L3 - Connectivity issue

Apparently we're all facing the same issue. The fact is, Neutron is too much complex...

Back with Havana, we faced this: https://bugs.launchpad.net/neutron/+bug/1252900

But now, with Ubuntu 14.04 + IceHouse, seems to be a different problem.

"Neutron + GRE or VXLAN", is very hard to stabilize and doesn't scale very well...

I'm seeing that it works better with "VLAN / Flat Networks" (i.e. without using the Neutron Network Node as Instance's default gateway). I'll try it instead...

Maybe James Denton, from Rackspace, might help us again!     :-P

Best!
Thiago

On 20 May 2014 11:21, Jason Bishop <jason.bishop at gmail.com<mailto:jason.bishop at gmail.com>> wrote:

that happened to me too.  maybe same root cause?

see http://ubuntuforums.org/showthread.php?t=2223789

cheers
jason


On Mon, May 19, 2014 at 10:59 PM, Martinx - ジェームズ <thiagocmartinsc at gmail.com<mailto:thiagocmartinsc at gmail.com>> wrote:
I managed to login into a Ubuntu Instance via `SPICE Console` and there is more network outages from within it, take a look ("apt-get update" froze too):

--
http://i.imgur.com/IeEXIjR.png
--

Am I missing something?!

On 20 May 2014 02:47, Martinx - ジェームズ <thiagocmartinsc at gmail.com<mailto:thiagocmartinsc at gmail.com>> wrote:
Hey Stackers!

I'm facing a weird network issue here, with IceHouse and Neutron L3... Using "Per-Project Router with Private Networks"...

I am unable to connect via SSH into an Instance, from its own "Project Namespace Router", look:

--
root at net-controller-1:~# ip netns exec qrouter-c252886a-1fd0-45dd-9e3b-4233786e7960 ssh -i ~/pem ubuntu at 192.168.1.17<mailto:ubuntu at 192.168.1.17> -v
OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to 192.168.1.17 [192.168.1.17] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: identity file /root/pem type -1
debug1: identity file /root/pem-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.6.1p1 Ubuntu-2ubuntu2
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6p1 Ubuntu-2ubuntu1
debug1: match: OpenSSH_6.6p1 Ubuntu-2ubuntu1 pat OpenSSH_6.5*,OpenSSH_6.6* compat 0x14000000
debug1: SSH2_MSG_KEXINIT sent
...
...
...ssh stucked!!!
...
--

The SSH connection froze at "debug1: SSH2_MSG_KEXINIT sent" and it never establishes...

From its attached `Floating IP`, the symptom is the same, I can't connect via SSH into a Instance, I tried it with 12.04 and 14.04, same result for both...

The Instance looks good from SPICE Consoles (but I need to login with "ubuntu user" via network first, using ssh key).

-

I'm running tcpdump on both "qr-$INT" and at the Compute Node DATA NETWORK, to see the GREv0 traffic too, here it is:

--
root at net-controller-1:~# ip netns exec qrouter-c252886a-1fd0-45dd-9e3b-4233786e7960 tcpdump -v -ni qr-eeb0d2f3-42

http://paste.openstack.org/show/80969/

NOTE: From the line ~16 to 32, SSH was already frozen...
--

root at compute-node-1:~# tcpdump -ni eth1 | grep -v IP6

http://paste.openstack.org/show/80970/
--

What can I do to deep investigate this?!

I double checked everything, including OVS bridges, sysctl.conf, `ethtool --offload ethX gro off` and etc... Don't know what's is wrong...

I think that this is the last problem I'm facing with IceHouse, I would like to put it into prod but, I am unable to do it right now...   :-/

Tks in advance!

Best,
Thiago


_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to     : openstack at lists.openstack.org<mailto:openstack at lists.openstack.org>
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20140520/ecf73d16/attachment.html>