Open Stack

Hi all,

Next week I'm doing an internal OpenStack training for my collegues,
and since we have an OpenStack installation already up&running, I
thought it would be easier to have them setup an openstack cloud
*inside* our openstack testbed. However, I'm testing my guide[1] and I'm
having troubles...

For the sake of clarity, let's call "host-OO" the production
OpenStack, and "guest-OO" the openstack created using VMs in the
"host-OO". Similarly, we call "guest-compute" the compute node of the
guest-OO, (which is a VM in host-OO), and guest-VM a VM of the
guest-OO, so a VM running on guest-compute. If you get lost, pull out
your totem[2] :)

VMs in the host-OO have two interfaces: one in an external network,
accessible from our laptopts, and the other is an OpenStack internal
network (vxlan).

Guest-OO instead uses gre for tenant networks.

The Guest-OO cloud is configured, everything seems correctly
configured, but I have no connectivity between the guest-VM and the
guest-neutron node (VM of host-OO running neutron for guest-OO).

The guest-VM doesn't get an IP via dhcp, and when I configure it
manually and try to ping the ip address of the dhcp interface in
guest-neutron, I see the gre tunnel in the guest-compute but nothing
on the guest-neutron. Here is a tcpdump from the guest-compute:

    root at compute-1:~# tcpdump -i any -n -v host 10.0.0.29  and ! tcp port 9696
    tcpdump: listening on any, link-type LINUX_SLL (Linux cooked),
capture size 65535 bytes
    23:41:38.397703 IP (tos 0x0, ttl 64, id 482, offset 0, flags [DF],
proto GRE (47), length 70)
        10.0.0.30 > 10.0.0.29: GREv0, Flags [key present], key=0x1, length 50
        ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.99.0.3
tell 10.99.0.4, length 28
    23:41:39.397619 IP (tos 0x0, ttl 64, id 483, offset 0, flags [DF],
proto GRE (47), length 70)
        10.0.0.30 > 10.0.0.29: GREv0, Flags [key present], key=0x1, length 50
        ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.99.0.3
tell 10.99.0.4, length 28

where 10.0.0.29 is the guest-neutron internal IP, 10.0.0.30 is the
guest-compute internal IP, 10.99.0.3 is the IP of the guest-VM and
10.99.0.4 the IP of the dhcp server on guest-neutron.

On the neutron node, however, tcpdump doesn't show these packets.

ovs-vsctl show on the guest-compute node seems fine:

    cf38508b-cbdb-4961-b42e-f58e713c27cf
        Bridge br-int
            fail_mode: secure
            Port "qvo7ae53e8e-f9"
                tag: 2
                Interface "qvo7ae53e8e-f9"
            Port patch-tun
                Interface patch-tun
                    type: patch
                    options: {peer=patch-int}
            Port br-int
                Interface br-int
                    type: internal
        Bridge br-tun
            Port "gre-0a00001d"
                Interface "gre-0a00001d"
                    type: gre
                    options: {df_default="true", in_key=flow,
local_ip="10.0.0.30", out_key=flow, remote_ip="10.0.0.29"}
            Port br-tun
                Interface br-tun
                    type: internal
            Port patch-int
                Interface patch-int
                    type: patch
                    options: {peer=patch-tun}
        ovs_version: "2.0.2"

and on the guest-neutron

    cc51b98d-4dfc-4ce1-811f-f20de9e376f6
        Bridge br-tun
            Port br-tun
                Interface br-tun
                    type: internal
            Port patch-int
                Interface patch-int
                    type: patch
                    options: {peer=patch-tun}
            Port "gre-0a00001e"
                Interface "gre-0a00001e"
                    type: gre
                    options: {df_default="true", in_key=flow,
local_ip="10.0.0.29", out_key=flow, remote_ip="10.0.0.30"}
        Bridge br-int
            fail_mode: secure
            Port br-int
                Interface br-int
                    type: internal
            Port "tapcdd5b8ec-ad"
                tag: 1
                Interface "tapcdd5b8ec-ad"
                    type: internal
            Port patch-tun
                Interface patch-tun
                    type: patch
                    options: {peer=patch-int}
        ovs_version: "2.0.2"

guest-compute and guest-neutron can of course ping each other.

I suspected an MTU issue, but mtu on the internal interface of
guest-compute and guest-neutron is already 1450, and I've tried
setting the mtu of the guest-VM to 1400, without success.

On the host-compute where guest-compute is running I can see the
packets when the guest-compute is pinging the guest-neutron:

    23:45:30.421830 IP (tos 0x0, ttl 64, id 12813, offset 0, flags
[DF], proto UDP (17), length 134)
        192.168.161.193.53785 > 192.168.161.194.4789: VXLAN, flags [I]
(0x08), vni 65537
    IP (tos 0x0, ttl 64, id 15138, offset 0, flags [none], proto ICMP
(1), length 84)
        10.0.0.29 > 10.0.0.30: ICMP echo reply, id 6979, seq 5, length 64

but when the guest-VM is pinging the guest-neutron I can't see any
packet coming.

I wonder if there is any known issue in encapsulating gre tunnels over
vxlan tunnels, or if I did something wrong with my installation (guest
or host...)

Thank you in advance, and sorry for the inception-like question :)

.a.

[1]: https://github.com/uzh/openstack-tutorial/tree/uzh-2015-may
[2]: http://inception.wikia.com/wiki/Totem

"Dreams feel real while we're in them. It's only when we wake up that
we realize something was actually strange. "

-- 
antonio.s.messina at gmail.com
antonio.messina at uzh.ch                     +41 (0)44 635 42 22
S3IT: Service and Support for Science IT   http://www.s3it.uzh.ch/
University of Zurich
Winterthurerstrasse 190
CH-8057 Zurich Switzerland