[Openstack] Rebooted, now can't ping my guest
Aaron Rosen
arosen at nicira.com
Thu Feb 28 04:15:44 UTC 2013
On Wed, Feb 27, 2013 at 7:11 PM, Chuan-Heng Hsiao
<hsiao.chuanheng at gmail.com> wrote:
> Hi Aaron and The King in Yellow,
>
> I also experienced this problem yesterday and I just solved.
>
> Besides ensuring quantum-openvswitch-agent processes are all up
> (except the controller node),
> there is a typo in the network node in basic install doc:
>
> http://docs.openstack.org/folsom/basic-install/content/basic-install_network.html
>
> DO NOT SET GATEWAY as 192.168.0.254.
Why do you say that. I don't see any problem with what the doc says:
# Management Network
auto eth0
iface eth0 inet static
address 192.168.0.2
netmask 255.255.255.0
gateway 192.168.0.254
dns-nameservers 8.8.8.8
# Data Network
auto eth1
iface eth1 inet static
address 10.10.10.1
netmask 255.255.255.0
# Public Bridge
auto eth2
iface eth2 inet manual
up ifconfig $IFACE 0.0.0.0 up
up ip link set $IFACE promisc on
down ifconfig $IFACE down
>
> or the routing table would be messed up.
>
>
> Also, if you want to do ssh, based on the following doc:
>
> https://github.com/mseknibilel/OpenStack-Folsom-Install-guide/blob/stable/GRE/OpenStack_Folsom_Install_Guide_WebVersion.rst
>
> REMEMBER TO DO THE FOLLOWING IN YOUR CONTROLLER NODE:
>
> route add -net 10.10.10.0/24 gw $router_proj_one_IP
>
> Sincerely,
> Hsiao
>
>
> On Thu, Feb 28, 2013 at 7:57 AM, Aaron Rosen <arosen at nicira.com> wrote:
>> Hi
>>
>> response inline
>>
>> On Wed, Feb 27, 2013 at 3:22 PM, The King in Yellow
>> <yellowking at gmail.com> wrote:
>>>
>>> I have been working on creating an OpenStack environment according to the
>>> Basic Install doc. It was working fine last night! In order to make sure I
>>> didn't mess anything up, I downed controller/network/compute nodes and
>>> cloned them (they are nested on ESXi 5.0u1).
>>>
>>> Upon coming back up, I can't ping my guests. I'm on the network node,
>>> pinging 10.5.5.3, which is a running guest. I'm guessing the GRE tunnel
>>> isn't coming between the compute and network node, since the br-* interfaces
>>> down? (After this, I manually "ip link set up" all br-* interfaces on both
>>> compute and network-- nothing)
>>>
>>> I have no experience with either Quantum or Open vSwitch, so I don't know
>>> what this is telling me. I'm rather at a loss-- can anybody point me in the
>>> right direction here? I don't see anything in the quantum logs right now
>>> that seems to indicate an error-- openvswitch-agent.log is cycling through
>>> things like the following, though:
>>>
>>> 2013-02-27 18:19:43 DEBUG [quantum.agent.linux.utils] Running command:
>>> sudo /usr/bin/quantum-rootwrap /etc/quantum/rootwrap.conf ovs-vsctl
>>> --timeout=2 get Interface qr-9f9041ce-65 external_ids
>>> 2013-02-27 18:19:43 DEBUG [quantum.agent.linux.utils]
>>> Command: ['sudo', '/usr/bin/quantum-rootwrap', '/etc/quantum/rootwrap.conf',
>>> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'qr-9f9041ce-65',
>>> 'external_ids']
>>> Exit code: 0
>>> Stdout: '{attached-mac="fa:16:3e:e2:38:da",
>>> iface-id="9f9041ce-654d-4706-a208-60cf5fca5d90", iface-status=active}\n'
>>> Stderr: ''
>>> 2013-02-27 18:19:43 DEBUG [quantum.agent.linux.utils] Running command:
>>> sudo /usr/bin/quantum-rootwrap /etc/quantum/rootwrap.conf ovs-vsctl
>>> --timeout=2 get Interface tap45ffdc5f-da external_ids
>>> 2013-02-27 18:19:43 DEBUG [quantum.agent.linux.utils]
>>> Command: ['sudo', '/usr/bin/quantum-rootwrap', '/etc/quantum/rootwrap.conf',
>>> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'tap45ffdc5f-da',
>>> 'external_ids']
>>> Exit code: 0
>>> Stdout: '{attached-mac="fa:16:3e:36:2e:54",
>>> iface-id="45ffdc5f-dad9-444a-aff4-3d39b607f828", iface-status=active}\n'
>>> Stderr: ''
>>> 2013-02-27 18:19:45 DEBUG [quantum.agent.linux.utils] Running command:
>>> sudo /usr/bin/quantum-rootwrap /etc/quantum/rootwrap.conf ovs-vsctl
>>> --timeout=2 list-ports br-int
>>> 2013-02-27 18:19:45 DEBUG [quantum.agent.linux.utils]
>>> Command: ['sudo', '/usr/bin/quantum-rootwrap', '/etc/quantum/rootwrap.conf',
>>> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int']
>>> Exit code: 0
>>> Stdout: 'patch-tun\nqr-9f9041ce-65\ntap45ffdc5f-da\n'
>>> Stderr: ''
>>> 2013-02-27 18:19:45 DEBUG [quantum.agent.linux.utils] Running command:
>>> sudo /usr/bin/quantum-rootwrap /etc/quantum/rootwrap.conf ovs-vsctl
>>> --timeout=2 get Interface patch-tun external_ids
>>> 2013-02-27 18:19:45 DEBUG [quantum.agent.linux.utils]
>>> Command: ['sudo', '/usr/bin/quantum-rootwrap', '/etc/quantum/rootwrap.conf',
>>> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'patch-tun', 'external_ids']
>>> Exit code: 0
>>> Stdout: '{}\n'
>>> Stderr: ''
>>>
>>>
>>> Here are the output of ifconfig -a, ovs-vsctl show, and ovs-cfctl of each
>>> bridge on the network node:
>>>
>>> root at os-network:~# ifconfig -a
>>> br-ex Link encap:Ethernet HWaddr 00:50:56:81:66:d8
>>> BROADCAST MULTICAST MTU:1500 Metric:1
>>> RX packets:23 errors:0 dropped:0 overruns:0 frame:0
>>> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>> collisions:0 txqueuelen:0
>>> RX bytes:1380 (1.3 KB) TX bytes:0 (0.0 B)
>>>
>>> br-int Link encap:Ethernet HWaddr 5e:5a:c3:07:44:42
>>> BROADCAST MULTICAST MTU:1500 Metric:1
>>> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>> collisions:0 txqueuelen:0
>>> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
>>>
>>> br-tun Link encap:Ethernet HWaddr 56:2d:9f:6c:ac:4f
>>> BROADCAST MULTICAST MTU:1500 Metric:1
>>> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>> collisions:0 txqueuelen:0
>>> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
>>>
>>> eth0 Link encap:Ethernet HWaddr 00:50:56:81:28:f4
>>> inet addr:192.168.0.2 Bcast:192.168.0.255 Mask:255.255.255.0
>>> inet6 addr: fe80::250:56ff:fe81:28f4/64 Scope:Link
>>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>>> RX packets:535 errors:0 dropped:10 overruns:0 frame:0
>>> TX packets:554 errors:0 dropped:0 overruns:0 carrier:0
>>> collisions:0 txqueuelen:1000
>>> RX bytes:137612 (137.6 KB) TX bytes:108783 (108.7 KB)
>>>
>>> eth1 Link encap:Ethernet HWaddr 00:50:56:81:44:e7
>>> inet addr:10.10.10.1 Bcast:10.10.10.255 Mask:255.255.255.0
>>> inet6 addr: fe80::250:56ff:fe81:44e7/64 Scope:Link
>>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>>> RX packets:76 errors:0 dropped:9 overruns:0 frame:0
>>> TX packets:40 errors:0 dropped:0 overruns:0 carrier:0
>>> collisions:0 txqueuelen:1000
>>> RX bytes:14531 (14.5 KB) TX bytes:7529 (7.5 KB)
>>>
>>> eth2 Link encap:Ethernet HWaddr 00:50:56:81:66:d8
>>> inet6 addr: fe80::250:56ff:fe81:66d8/64 Scope:Link
>>> UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
>>> RX packets:24 errors:0 dropped:0 overruns:0 frame:0
>>> TX packets:15 errors:0 dropped:0 overruns:0 carrier:0
>>> collisions:0 txqueuelen:1000
>>> RX bytes:1440 (1.4 KB) TX bytes:3496 (3.4 KB)
>>>
>>> eth3 Link encap:Ethernet HWaddr 00:50:56:81:08:97
>>> inet addr:10.42.39.253 Bcast:10.42.39.255 Mask:255.255.254.0
>>> inet6 addr: fe80::250:56ff:fe81:897/64 Scope:Link
>>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>>> RX packets:500 errors:0 dropped:0 overruns:0 frame:0
>>> TX packets:530 errors:0 dropped:0 overruns:0 carrier:0
>>> collisions:0 txqueuelen:1000
>>> RX bytes:52288 (52.2 KB) TX bytes:59695 (59.6 KB)
>>>
>>> lo Link encap:Local Loopback
>>> inet addr:127.0.0.1 Mask:255.0.0.0
>>> inet6 addr: ::1/128 Scope:Host
>>> UP LOOPBACK RUNNING MTU:16436 Metric:1
>>> RX packets:231 errors:0 dropped:0 overruns:0 frame:0
>>> TX packets:231 errors:0 dropped:0 overruns:0 carrier:0
>>> collisions:0 txqueuelen:0
>>> RX bytes:22595 (22.5 KB) TX bytes:22595 (22.5 KB)
>>>
>>> qg-28108125-11 Link encap:Ethernet HWaddr 6a:5e:96:0e:59:a0
>>> inet addr:10.42.36.130 Bcast:10.42.37.255 Mask:255.255.254.0
>>> BROADCAST MULTICAST MTU:1500 Metric:1
>>> RX packets:23 errors:0 dropped:0 overruns:0 frame:0
>>> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>> collisions:0 txqueuelen:0
>>> RX bytes:1380 (1.3 KB) TX bytes:0 (0.0 B)
>>>
>>> qr-9f9041ce-65 Link encap:Ethernet HWaddr 26:0e:04:69:99:1a
>>> inet addr:10.5.5.1 Bcast:10.5.5.255 Mask:255.255.255.0
>>> BROADCAST MULTICAST MTU:1500 Metric:1
>>> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>> collisions:0 txqueuelen:0
>>> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
>>>
>>> tap45ffdc5f-da Link encap:Ethernet HWaddr 0e:bd:f3:93:4b:b0
>>> inet addr:10.5.5.2 Bcast:10.5.5.255 Mask:255.255.255.0
>>> BROADCAST MULTICAST MTU:1500 Metric:1
>>> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>> collisions:0 txqueuelen:0
>>> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
>>>
>>> root at os-network:~# ovs-vsctl show
>>> e232f8c8-1cb8-4cf5-9de5-49f41e59fd38
>>> Bridge br-int
>>> Port "qr-9f9041ce-65"
>>> tag: 1
>>> Interface "qr-9f9041ce-65"
>>> type: internal
>>> Port patch-tun
>>> Interface patch-tun
>>> type: patch
>>> options: {peer=patch-int}
>>> Port br-int
>>> Interface br-int
>>> type: internal
>>> Port "tap45ffdc5f-da"
>>> tag: 1
>>> Interface "tap45ffdc5f-da"
>>> type: internal
>>> Bridge br-ex
>>> Port br-ex
>>> Interface br-ex
>>> type: internal
>>> Port "qg-28108125-11"
>>> Interface "qg-28108125-11"
>>> type: internal
>>> Port "eth2"
>>> Interface "eth2"
>>> Bridge br-tun
>>> Port "gre-2"
>>> Interface "gre-2"
>>> type: gre
>>
>> Can you ensure that the quantum-openvswitch-agent process is running
>> on all your your nodes after the reboot. You should have a tunnel
>> here between each machine.
>>
>>> options: {in_key=flow, out_key=flow, remote_ip="10.10.10.2"}
>>> Port patch-int
>>> Interface patch-int
>>> type: patch
>>> options: {peer=patch-tun}
>>> Port br-tun
>>> Interface br-tun
>>> type: internal
>>> ovs_version: "1.4.0+build0"
>>> root at os-network:~# ovs-ofctl show br-int
>>> OFPT_FEATURES_REPLY (xid=0x1): ver:0x1, dpid:00005e5ac3074442
>>> n_tables:255, n_buffers:256
>>> features: capabilities:0xc7, actions:0xfff
>>> 1(qr-9f9041ce-65): addr:26:0e:04:69:99:1a
>>> config: PORT_DOWN
>>> state: LINK_DOWN
>>> 2(tap45ffdc5f-da): addr:0e:bd:f3:93:4b:b0
>>> config: PORT_DOWN
>>> state: LINK_DOWN
>>> 3(patch-tun): addr:6a:b6:66:41:31:54
>>> config: 0
>>> state: 0
>>> LOCAL(br-int): addr:5e:5a:c3:07:44:42
>>> config: PORT_DOWN
>>> state: LINK_DOWN
>>> OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0
>>> root at os-network:~# ovs-ofctl show br-ex
>>> OFPT_FEATURES_REPLY (xid=0x1): ver:0x1, dpid:00000050568166d8
>>> n_tables:255, n_buffers:256
>>> features: capabilities:0xc7, actions:0xfff
>>> 1(eth2): addr:00:50:56:81:66:d8
>>> config: 0
>>> state: 0
>>> current: 10GB-FD COPPER
>>> advertised: COPPER
>>> supported: 1GB-FD 10GB-FD COPPER
>>> 2(qg-28108125-11): addr:6a:5e:96:0e:59:a0
>>> config: PORT_DOWN
>>> state: LINK_DOWN
>>> LOCAL(br-ex): addr:00:50:56:81:66:d8
>>> config: PORT_DOWN
>>> state: LINK_DOWN
>>> OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0
>>> root at os-network:~# ovs-ofctl show br-tun
>>> OFPT_FEATURES_REPLY (xid=0x1): ver:0x1, dpid:0000562d9f6cac4f
>>> n_tables:255, n_buffers:256
>>> features: capabilities:0xc7, actions:0xfff
>>> 1(patch-int): addr:8e:c3:2b:55:e5:73
>>> config: 0
>>> state: 0
>>> 2(gre-2): addr:5a:73:ff:6b:16:37
>>> config: 0
>>> state: 0
>>> LOCAL(br-tun): addr:56:2d:9f:6c:ac:4f
>>> config: PORT_DOWN
>>> state: LINK_DOWN
>>> OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0
>>> root at os-network:~#
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~openstack
>>> Post to : openstack at lists.launchpad.net
>>> Unsubscribe : https://launchpad.net/~openstack
>>> More help : https://help.launchpad.net/ListHelp
>>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack at lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help : https://help.launchpad.net/ListHelp
More information about the Openstack
mailing list