<div dir="ltr"><div><div><div><div><br></div>I have been working on creating an OpenStack environment according to the <a href="http://docs.openstack.org/folsom/basic-install/content/index.html">Basic Install</a> doc. It was working fine last night! In order to make sure I didn't mess anything up, I downed controller/network/compute nodes and cloned them (they are nested on ESXi 5.0u1).<br>
<br></div>Upon coming back up, I can't ping my guests. I'm on the network node, pinging 10.5.5.3, which is a running guest. I'm guessing the GRE tunnel isn't coming between the compute and network node, since the br-* interfaces down? (After this, I manually "ip link set up" all br-* interfaces on both compute and network-- nothing)<br>
<br></div>I have no experience with either Quantum or Open vSwitch, so I don't know what this is telling me. I'm rather at a loss-- can anybody point me in the right direction here? I don't see anything in the quantum logs right now that seems to indicate an error-- openvswitch-agent.log is cycling through things like the following, though:<br>
<br><span style="font-family:courier new,monospace">2013-02-27 18:19:43 DEBUG [quantum.agent.linux.utils] Running command: sudo /usr/bin/quantum-rootwrap /etc/quantum/rootwrap.conf ovs-vsctl --timeout=2 get Interface qr-9f9041ce-65 external_ids<br>
2013-02-27 18:19:43 DEBUG [quantum.agent.linux.utils] <br>Command: ['sudo', '/usr/bin/quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'qr-9f9041ce-65', 'external_ids']<br>
Exit code: 0<br>Stdout: '{attached-mac="fa:16:3e:e2:38:da", iface-id="9f9041ce-654d-4706-a208-60cf5fca5d90", iface-status=active}\n'<br>Stderr: ''<br>2013-02-27 18:19:43 DEBUG [quantum.agent.linux.utils] Running command: sudo /usr/bin/quantum-rootwrap /etc/quantum/rootwrap.conf ovs-vsctl --timeout=2 get Interface tap45ffdc5f-da external_ids<br>
2013-02-27 18:19:43 DEBUG [quantum.agent.linux.utils] <br>Command: ['sudo', '/usr/bin/quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'tap45ffdc5f-da', 'external_ids']<br>
Exit code: 0<br>Stdout: '{attached-mac="fa:16:3e:36:2e:54", iface-id="45ffdc5f-dad9-444a-aff4-3d39b607f828", iface-status=active}\n'<br>Stderr: ''<br>2013-02-27 18:19:45 DEBUG [quantum.agent.linux.utils] Running command: sudo /usr/bin/quantum-rootwrap /etc/quantum/rootwrap.conf ovs-vsctl --timeout=2 list-ports br-int<br>
2013-02-27 18:19:45 DEBUG [quantum.agent.linux.utils] <br>Command: ['sudo', '/usr/bin/quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int']<br>
Exit code: 0<br>Stdout: 'patch-tun\nqr-9f9041ce-65\ntap45ffdc5f-da\n'<br>Stderr: ''<br>2013-02-27 18:19:45 DEBUG [quantum.agent.linux.utils] Running command: sudo /usr/bin/quantum-rootwrap /etc/quantum/rootwrap.conf ovs-vsctl --timeout=2 get Interface patch-tun external_ids<br>
2013-02-27 18:19:45 DEBUG [quantum.agent.linux.utils] <br>Command: ['sudo', '/usr/bin/quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'patch-tun', 'external_ids']<br>
Exit code: 0<br>Stdout: '{}\n'<br>Stderr: ''<br></span><br><br></div>Here are the output of ifconfig -a, ovs-vsctl show, and ovs-cfctl of each bridge on the network node:<br><div><div><div><br><span style="font-family:courier new,monospace">root@os-network:~# ifconfig -a<br>
br-ex Link encap:Ethernet HWaddr 00:50:56:81:66:d8 <br> BROADCAST MULTICAST MTU:1500 Metric:1<br> RX packets:23 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0<br>
collisions:0 txqueuelen:0 <br> RX bytes:1380 (1.3 KB) TX bytes:0 (0.0 B)<br><br>br-int Link encap:Ethernet HWaddr 5e:5a:c3:07:44:42 <br> BROADCAST MULTICAST MTU:1500 Metric:1<br> RX packets:0 errors:0 dropped:0 overruns:0 frame:0<br>
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:0 <br> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)<br><br>br-tun Link encap:Ethernet HWaddr 56:2d:9f:6c:ac:4f <br>
BROADCAST MULTICAST MTU:1500 Metric:1<br> RX packets:0 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:0 <br>
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)<br><br>eth0 Link encap:Ethernet HWaddr 00:50:56:81:28:f4 <br> inet addr:192.168.0.2 Bcast:192.168.0.255 Mask:255.255.255.0<br> inet6 addr: fe80::250:56ff:fe81:28f4/64 Scope:Link<br>
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1<br> RX packets:535 errors:0 dropped:10 overruns:0 frame:0<br> TX packets:554 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:1000 <br>
RX bytes:137612 (137.6 KB) TX bytes:108783 (108.7 KB)<br><br>eth1 Link encap:Ethernet HWaddr 00:50:56:81:44:e7 <br> inet addr:10.10.10.1 Bcast:10.10.10.255 Mask:255.255.255.0<br> inet6 addr: fe80::250:56ff:fe81:44e7/64 Scope:Link<br>
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1<br> RX packets:76 errors:0 dropped:9 overruns:0 frame:0<br> TX packets:40 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:1000 <br>
RX bytes:14531 (14.5 KB) TX bytes:7529 (7.5 KB)<br><br>eth2 Link encap:Ethernet HWaddr 00:50:56:81:66:d8 <br> inet6 addr: fe80::250:56ff:fe81:66d8/64 Scope:Link<br> UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1<br>
RX packets:24 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:15 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:1000 <br> RX bytes:1440 (1.4 KB) TX bytes:3496 (3.4 KB)<br>
<br>eth3 Link encap:Ethernet HWaddr 00:50:56:81:08:97 <br> inet addr:10.42.39.253 Bcast:10.42.39.255 Mask:255.255.254.0<br> inet6 addr: fe80::250:56ff:fe81:897/64 Scope:Link<br> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1<br>
RX packets:500 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:530 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:1000 <br> RX bytes:52288 (52.2 KB) TX bytes:59695 (59.6 KB)<br>
<br>lo Link encap:Local Loopback <br> inet addr:127.0.0.1 Mask:255.0.0.0<br> inet6 addr: ::1/128 Scope:Host<br> UP LOOPBACK RUNNING MTU:16436 Metric:1<br> RX packets:231 errors:0 dropped:0 overruns:0 frame:0<br>
TX packets:231 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:0 <br> RX bytes:22595 (22.5 KB) TX bytes:22595 (22.5 KB)<br><br>qg-28108125-11 Link encap:Ethernet HWaddr 6a:5e:96:0e:59:a0 <br>
inet addr:10.42.36.130 Bcast:10.42.37.255 Mask:255.255.254.0<br> BROADCAST MULTICAST MTU:1500 Metric:1<br> RX packets:23 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0<br>
collisions:0 txqueuelen:0 <br> RX bytes:1380 (1.3 KB) TX bytes:0 (0.0 B)<br><br>qr-9f9041ce-65 Link encap:Ethernet HWaddr 26:0e:04:69:99:1a <br> inet addr:10.5.5.1 Bcast:10.5.5.255 Mask:255.255.255.0<br>
BROADCAST MULTICAST MTU:1500 Metric:1<br> RX packets:0 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:0 <br>
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)<br><br>tap45ffdc5f-da Link encap:Ethernet HWaddr 0e:bd:f3:93:4b:b0 <br> inet addr:10.5.5.2 Bcast:10.5.5.255 Mask:255.255.255.0<br> BROADCAST MULTICAST MTU:1500 Metric:1<br>
RX packets:0 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:0 <br> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)<br>
<br>root@os-network:~# ovs-vsctl show<br>e232f8c8-1cb8-4cf5-9de5-49f41e59fd38<br> Bridge br-int<br> Port "qr-9f9041ce-65"<br> tag: 1<br> Interface "qr-9f9041ce-65"<br> type: internal<br>
Port patch-tun<br> Interface patch-tun<br> type: patch<br> options: {peer=patch-int}<br> Port br-int<br> Interface br-int<br> type: internal<br>
Port "tap45ffdc5f-da"<br> tag: 1<br> Interface "tap45ffdc5f-da"<br> type: internal<br> Bridge br-ex<br> Port br-ex<br> Interface br-ex<br>
type: internal<br> Port "qg-28108125-11"<br> Interface "qg-28108125-11"<br> type: internal<br> Port "eth2"<br> Interface "eth2"<br>
Bridge br-tun<br> Port "gre-2"<br> Interface "gre-2"<br> type: gre<br> options: {in_key=flow, out_key=flow, remote_ip="10.10.10.2"}<br> Port patch-int<br>
Interface patch-int<br> type: patch<br> options: {peer=patch-tun}<br> Port br-tun<br> Interface br-tun<br> type: internal<br> ovs_version: "1.4.0+build0"<br>
root@os-network:~# ovs-ofctl show br-int<br>OFPT_FEATURES_REPLY (xid=0x1): ver:0x1, dpid:00005e5ac3074442<br>n_tables:255, n_buffers:256<br>features: capabilities:0xc7, actions:0xfff<br> 1(qr-9f9041ce-65): addr:26:0e:04:69:99:1a<br>
config: PORT_DOWN<br> state: LINK_DOWN<br> 2(tap45ffdc5f-da): addr:0e:bd:f3:93:4b:b0<br> config: PORT_DOWN<br> state: LINK_DOWN<br> 3(patch-tun): addr:6a:b6:66:41:31:54<br> config: 0<br>
state: 0<br> LOCAL(br-int): addr:5e:5a:c3:07:44:42<br> config: PORT_DOWN<br> state: LINK_DOWN<br>OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0<br>root@os-network:~# ovs-ofctl show br-ex<br>
OFPT_FEATURES_REPLY (xid=0x1): ver:0x1, dpid:00000050568166d8<br>n_tables:255, n_buffers:256<br>features: capabilities:0xc7, actions:0xfff<br> 1(eth2): addr:00:50:56:81:66:d8<br> config: 0<br> state: 0<br>
current: 10GB-FD COPPER<br> advertised: COPPER<br> supported: 1GB-FD 10GB-FD COPPER<br> 2(qg-28108125-11): addr:6a:5e:96:0e:59:a0<br> config: PORT_DOWN<br> state: LINK_DOWN<br> LOCAL(br-ex): addr:00:50:56:81:66:d8<br>
config: PORT_DOWN<br> state: LINK_DOWN<br>OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0<br>root@os-network:~# ovs-ofctl show br-tun<br>OFPT_FEATURES_REPLY (xid=0x1): ver:0x1, dpid:0000562d9f6cac4f<br>
n_tables:255, n_buffers:256<br>features: capabilities:0xc7, actions:0xfff<br> 1(patch-int): addr:8e:c3:2b:55:e5:73<br> config: 0<br> state: 0<br> 2(gre-2): addr:5a:73:ff:6b:16:37<br> config: 0<br>
state: 0<br> LOCAL(br-tun): addr:56:2d:9f:6c:ac:4f<br> config: PORT_DOWN<br> state: LINK_DOWN<br>OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0<br>root@os-network:~# </span><br></div>
</div></div></div>