<font size=2 face="sans-serif">I am consistently suffering a network problem
in simple DevStack installations. Am I doing something wrong, or
is this a bug, or is it to be expected?</font>
<br>
<br><font size=2 face="sans-serif">I install DevStack, using a pretty basic
local.conf; the only thing it says that is relevant to networking is setting
HOST_IP to the address of the machine where I am installing DevStack. Thus,
it is using nova networking (the default), with the default address ranges.
DevStack completes successfully. I edit the default security
group, completely opening up ICMP, TCP, and UDP. I instantiate an
image. Using Horizon I log into the console of that image. From
that instance I can ping anywhere. Then I associate a floating IP
address with that instance. While that floating IP is associated,
I can not ping anywhere --- that instance can only ping the host's address
and those of other VMs on the same host, the instance can NOT ping other
hosts on the same subnet as the instance's host nor anything more distant.</font>
<br>
<br><font size=2 face="sans-serif">I get this both when installing DevStack
onto bare metal and when installing DevStack into a VM instance. I
get this when using branch stable/havana and when using the master branch
(over the last few weeks).</font>
<br>
<br><font size=2 face="sans-serif">Following are the details from an example
in which DevStack (master branch) was installed onto a bare metal machine
a few days ago. Before installing DevStack, the host's networking
config was as follows:</font>
<br>
<br><tt><font size=2>ubu_wa@pvespa015:~$ ifconfig</font></tt>
<br><tt><font size=2>eth0 Link encap:Ethernet HWaddr
00:21:5e:21:04:78 </font></tt>
<br><tt><font size=2> inet addr:9.0.0.191
Bcast:9.0.1.255 Mask:255.255.254.0</font></tt>
<br><tt><font size=2> inet6 addr: fe80::221:5eff:fe21:478/64
Scope:Link</font></tt>
<br><tt><font size=2> UP BROADCAST RUNNING
MULTICAST MTU:1500 Metric:1</font></tt>
<br><tt><font size=2> RX packets:266
errors:0 dropped:1 overruns:0 frame:0</font></tt>
<br><tt><font size=2> TX packets:147
errors:0 dropped:0 overruns:0 carrier:0</font></tt>
<br><tt><font size=2> collisions:0 txqueuelen:1000
</font></tt>
<br><tt><font size=2> RX bytes:24924
(24.9 KB) TX bytes:19720 (19.7 KB)</font></tt>
<br>
<br><tt><font size=2>eth1 Link encap:Ethernet HWaddr
00:21:5e:21:04:7a </font></tt>
<br><tt><font size=2> UP BROADCAST MULTICAST
MTU:1500 Metric:1</font></tt>
<br><tt><font size=2> RX packets:0 errors:0
dropped:0 overruns:0 frame:0</font></tt>
<br><tt><font size=2> TX packets:0 errors:0
dropped:0 overruns:0 carrier:0</font></tt>
<br><tt><font size=2> collisions:0 txqueuelen:1000
</font></tt>
<br><tt><font size=2> RX bytes:0 (0.0
B) TX bytes:0 (0.0 B)</font></tt>
<br>
<br><tt><font size=2>lo Link encap:Local Loopback
</font></tt>
<br><tt><font size=2> inet addr:127.0.0.1
Mask:255.0.0.0</font></tt>
<br><tt><font size=2> inet6 addr: ::1/128
Scope:Host</font></tt>
<br><tt><font size=2> UP LOOPBACK RUNNING
MTU:65536 Metric:1</font></tt>
<br><tt><font size=2> RX packets:23 errors:0
dropped:0 overruns:0 frame:0</font></tt>
<br><tt><font size=2> TX packets:23 errors:0
dropped:0 overruns:0 carrier:0</font></tt>
<br><tt><font size=2> collisions:0 txqueuelen:0
</font></tt>
<br><tt><font size=2> RX bytes:2841 (2.8
KB) TX bytes:2841 (2.8 KB)</font></tt>
<br>
<br><tt><font size=2>ubu_wa@pvespa015:~$ netstat -nr</font></tt>
<br><tt><font size=2>Kernel IP routing table</font></tt>
<br><tt><font size=2>Destination Gateway
Genmask Flags MSS Window irtt
Iface</font></tt>
<br><tt><font size=2>0.0.0.0 9.0.0.2
0.0.0.0 UG
0 0 0 eth0</font></tt>
<br><tt><font size=2>9.0.0.0 0.0.0.0
255.255.254.0 U
0 0 0 eth0</font></tt>
<br><tt><font size=2>169.254.0.0 0.0.0.0
255.255.0.0 U 0 0
0 eth0</font></tt>
<br><tt><font size=2>ubu_wa@pvespa015:~$</font></tt>
<br>
<br><font size=2 face="sans-serif">Here is the local.conf that I used:</font>
<br>
<br><tt><font size=2>[[local|localrc]]</font></tt>
<br><tt><font size=2>HOST_IP=9.0.0.191</font></tt>
<br><tt><font size=2>#SERVICE_HOST=FIXME</font></tt>
<br><tt><font size=2>ADMIN_PASSWORD=POK-1428</font></tt>
<br><tt><font size=2>ENABLED_SERVICES+=,heat,h-api,h-api-cfn,h-api-cw,h-eng</font></tt>
<br><tt><font size=2>MYSQL_PASSWORD=$ADMIN_PASSWORD</font></tt>
<br><tt><font size=2>RABBIT_PASSWORD=$ADMIN_PASSWORD</font></tt>
<br><tt><font size=2>SERVICE_PASSWORD=$ADMIN_PASSWORD</font></tt>
<br><tt><font size=2>DEST=/opt/stack</font></tt>
<br><tt><font size=2>LOGFILE=stack.sh.log</font></tt>
<br><tt><font size=2>LOGDAYS=7</font></tt>
<br><tt><font size=2>LOG_COLOR=False</font></tt>
<br><tt><font size=2>SCREEN_LOGDIR=$DEST/logs/screen</font></tt>
<br><tt><font size=2>RECLONE=yes</font></tt>
<br><tt><font size=2>KEYSTONE_CATALOG_BACKEND=sql</font></tt>
<br><tt><font size=2>VOLUME_GROUP="stack-volumes"</font></tt>
<br><tt><font size=2>VOLUME_NAME_PREFIX="volume-"</font></tt>
<br><tt><font size=2>VOLUME_BACKING_FILE_SIZE=5130M</font></tt>
<br><tt><font size=2>API_RATE_LIMIT=False</font></tt>
<br><tt><font size=2>IMAGE_URLS+=",</font></tt><a href="http://fedorapeople.org/groups/heat/prebuilt-jeos-images/F17-x86_64-cfntools.qcow2"><tt><font size=2>http://fedorapeople.org/groups/heat/prebuilt-jeos-images/F17-x86_64-cfntools.qcow2</font></tt></a><tt><font size=2>"</font></tt>
<br>
<br><font size=2 face="sans-serif">There were no failures reported from
the DevStack installation. After that installation, the host's network
config looked like this:</font>
<br>
<br><tt><font size=2>ubu_wa@pvespa015:~$ ifconfig</font></tt>
<br><tt><font size=2>eth0 Link encap:Ethernet HWaddr
00:21:5e:21:04:78 </font></tt>
<br><tt><font size=2> inet addr:9.0.0.191
Bcast:9.0.1.255 Mask:255.255.254.0</font></tt>
<br><tt><font size=2> inet6 addr: fe80::221:5eff:fe21:478/64
Scope:Link</font></tt>
<br><tt><font size=2> UP BROADCAST RUNNING
MULTICAST MTU:1500 Metric:1</font></tt>
<br><tt><font size=2> RX packets:677321
errors:0 dropped:161 overruns:0 frame:0</font></tt>
<br><tt><font size=2> TX packets:299006
errors:0 dropped:0 overruns:0 carrier:0</font></tt>
<br><tt><font size=2> collisions:0 txqueuelen:1000
</font></tt>
<br><tt><font size=2> RX bytes:858856772
(858.8 MB) TX bytes:20708280 (20.7 MB)</font></tt>
<br>
<br><tt><font size=2>eth1 Link encap:Ethernet HWaddr
00:21:5e:21:04:7a </font></tt>
<br><tt><font size=2> UP BROADCAST MULTICAST
MTU:1500 Metric:1</font></tt>
<br><tt><font size=2> RX packets:0 errors:0
dropped:0 overruns:0 frame:0</font></tt>
<br><tt><font size=2> TX packets:0 errors:0
dropped:0 overruns:0 carrier:0</font></tt>
<br><tt><font size=2> collisions:0 txqueuelen:1000
</font></tt>
<br><tt><font size=2> RX bytes:0 (0.0
B) TX bytes:0 (0.0 B)</font></tt>
<br>
<br><tt><font size=2>lo Link encap:Local Loopback
</font></tt>
<br><tt><font size=2> inet addr:127.0.0.1
Mask:255.0.0.0</font></tt>
<br><tt><font size=2> inet6 addr: ::1/128
Scope:Host</font></tt>
<br><tt><font size=2> UP LOOPBACK RUNNING
MTU:65536 Metric:1</font></tt>
<br><tt><font size=2> RX packets:277765
errors:0 dropped:0 overruns:0 frame:0</font></tt>
<br><tt><font size=2> TX packets:277765
errors:0 dropped:0 overruns:0 carrier:0</font></tt>
<br><tt><font size=2> collisions:0 txqueuelen:0
</font></tt>
<br><tt><font size=2> RX bytes:537924232
(537.9 MB) TX bytes:537924232 (537.9 MB)</font></tt>
<br>
<br><tt><font size=2>virbr0 Link encap:Ethernet HWaddr
3a:a3:37:44:79:f4 </font></tt>
<br><tt><font size=2> inet addr:192.168.122.1
Bcast:192.168.122.255 Mask:255.255.255.0</font></tt>
<br><tt><font size=2> UP BROADCAST MULTICAST
MTU:1500 Metric:1</font></tt>
<br><tt><font size=2> RX packets:0 errors:0
dropped:0 overruns:0 frame:0</font></tt>
<br><tt><font size=2> TX packets:0 errors:0
dropped:0 overruns:0 carrier:0</font></tt>
<br><tt><font size=2> collisions:0 txqueuelen:0
</font></tt>
<br><tt><font size=2> RX bytes:0 (0.0
B) TX bytes:0 (0.0 B)</font></tt>
<br>
<br><tt><font size=2>ubu_wa@pvespa015:~$ netstat -nr</font></tt>
<br><tt><font size=2>Kernel IP routing table</font></tt>
<br><tt><font size=2>Destination Gateway
Genmask Flags MSS Window irtt
Iface</font></tt>
<br><tt><font size=2>0.0.0.0 9.0.0.2
0.0.0.0 UG
0 0 0 eth0</font></tt>
<br><tt><font size=2>9.0.0.0 0.0.0.0
255.255.254.0 U
0 0 0 eth0</font></tt>
<br><tt><font size=2>169.254.0.0 0.0.0.0
255.255.0.0 U 0 0
0 eth0</font></tt>
<br><tt><font size=2>192.168.122.0 0.0.0.0
255.255.255.0 U 0 0
0 virbr0</font></tt>
<br><tt><font size=2>ubu_wa@pvespa015:~$</font></tt>
<br>
<br><font size=2 face="sans-serif">I then created and tested some VM instances.
After that, the host's network config looked like this:</font>
<br>
<br><tt><font size=2>ubu_wa@pvespa015:~$ ifconfig</font></tt>
<br><tt><font size=2>br100 Link encap:Ethernet HWaddr
00:21:5e:21:04:78 </font></tt>
<br><tt><font size=2> inet addr:10.0.0.1
Bcast:10.0.0.255 Mask:255.255.255.0</font></tt>
<br><tt><font size=2> inet6 addr: fe80::fce9:c9ff:feab:ac5/64
Scope:Link</font></tt>
<br><tt><font size=2> UP BROADCAST RUNNING
MULTICAST MTU:1500 Metric:1</font></tt>
<br><tt><font size=2> RX packets:2065
errors:0 dropped:2 overruns:0 frame:0</font></tt>
<br><tt><font size=2> TX packets:213
errors:0 dropped:0 overruns:0 carrier:0</font></tt>
<br><tt><font size=2> collisions:0 txqueuelen:0
</font></tt>
<br><tt><font size=2> RX bytes:123592
(123.5 KB) TX bytes:33219 (33.2 KB)</font></tt>
<br>
<br><tt><font size=2>eth0 Link encap:Ethernet HWaddr
00:21:5e:21:04:78 </font></tt>
<br><tt><font size=2> inet6 addr: fe80::221:5eff:fe21:478/64
Scope:Link</font></tt>
<br><tt><font size=2> UP BROADCAST RUNNING
MULTICAST MTU:1500 Metric:1</font></tt>
<br><tt><font size=2> RX packets:691620
errors:0 dropped:209 overruns:0 frame:0</font></tt>
<br><tt><font size=2> TX packets:303320
errors:0 dropped:0 overruns:0 carrier:0</font></tt>
<br><tt><font size=2> collisions:0 txqueuelen:1000
</font></tt>
<br><tt><font size=2> RX bytes:860180754
(860.1 MB) TX bytes:24551439 (24.5 MB)</font></tt>
<br>
<br><tt><font size=2>eth1 Link encap:Ethernet HWaddr
00:21:5e:21:04:7a </font></tt>
<br><tt><font size=2> UP BROADCAST MULTICAST
MTU:1500 Metric:1</font></tt>
<br><tt><font size=2> RX packets:0 errors:0
dropped:0 overruns:0 frame:0</font></tt>
<br><tt><font size=2> TX packets:0 errors:0
dropped:0 overruns:0 carrier:0</font></tt>
<br><tt><font size=2> collisions:0 txqueuelen:1000
</font></tt>
<br><tt><font size=2> RX bytes:0 (0.0
B) TX bytes:0 (0.0 B)</font></tt>
<br>
<br><tt><font size=2>lo Link encap:Local Loopback
</font></tt>
<br><tt><font size=2> inet addr:127.0.0.1
Mask:255.0.0.0</font></tt>
<br><tt><font size=2> inet6 addr: ::1/128
Scope:Host</font></tt>
<br><tt><font size=2> UP LOOPBACK RUNNING
MTU:65536 Metric:1</font></tt>
<br><tt><font size=2> RX packets:407649
errors:0 dropped:0 overruns:0 frame:0</font></tt>
<br><tt><font size=2> TX packets:407649
errors:0 dropped:0 overruns:0 carrier:0</font></tt>
<br><tt><font size=2> collisions:0 txqueuelen:0
</font></tt>
<br><tt><font size=2> RX bytes:1043746254
(1.0 GB) TX bytes:1043746254 (1.0 GB)</font></tt>
<br>
<br><tt><font size=2>virbr0 Link encap:Ethernet HWaddr
3a:a3:37:44:79:f4 </font></tt>
<br><tt><font size=2> inet addr:192.168.122.1
Bcast:192.168.122.255 Mask:255.255.255.0</font></tt>
<br><tt><font size=2> UP BROADCAST MULTICAST
MTU:1500 Metric:1</font></tt>
<br><tt><font size=2> RX packets:0 errors:0
dropped:0 overruns:0 frame:0</font></tt>
<br><tt><font size=2> TX packets:0 errors:0
dropped:0 overruns:0 carrier:0</font></tt>
<br><tt><font size=2> collisions:0 txqueuelen:0
</font></tt>
<br><tt><font size=2> RX bytes:0 (0.0
B) TX bytes:0 (0.0 B)</font></tt>
<br>
<br><tt><font size=2>vnet0 Link encap:Ethernet HWaddr
fe:16:3e:59:bf:df </font></tt>
<br><tt><font size=2> inet6 addr: fe80::fc16:3eff:fe59:bfdf/64
Scope:Link</font></tt>
<br><tt><font size=2> UP BROADCAST RUNNING
MULTICAST MTU:1500 Metric:1</font></tt>
<br><tt><font size=2> RX packets:75 errors:0
dropped:0 overruns:0 frame:0</font></tt>
<br><tt><font size=2> TX packets:1843
errors:0 dropped:0 overruns:0 carrier:0</font></tt>
<br><tt><font size=2> collisions:0 txqueuelen:500
</font></tt>
<br><tt><font size=2> RX bytes:7575 (7.5
KB) TX bytes:132432 (132.4 KB)</font></tt>
<br>
<br><tt><font size=2>ubu_wa@pvespa015:~$ netstat -nr</font></tt>
<br><tt><font size=2>Kernel IP routing table</font></tt>
<br><tt><font size=2>Destination Gateway
Genmask Flags MSS Window irtt
Iface</font></tt>
<br><tt><font size=2>0.0.0.0 9.0.0.2
0.0.0.0 UG
0 0 0 br100</font></tt>
<br><tt><font size=2>9.0.0.0 0.0.0.0
255.255.254.0 U
0 0 0 br100</font></tt>
<br><tt><font size=2>10.0.0.0 0.0.0.0
255.255.255.0 U
0 0 0 br100</font></tt>
<br><tt><font size=2>192.168.122.0 0.0.0.0
255.255.255.0 U 0 0
0 virbr0</font></tt>
<br><tt><font size=2>ubu_wa@pvespa015:~$</font></tt>
<br>
<br><font size=2 face="sans-serif">Following are some examples from an
instance of F17 with private IP address 10.0.0.2. While it has no
floating IP address, it can ping 10.0.0.2, 10.0.0.8 (a sibling VM), 9.0.0.191
(its host), 9.0.0.193 (another machine on the same subnet), and 8.8.8.8
(something entirely outside IBM's intranet). Here is what the network
config looks like inside the VM:</font>
<br>
<br><tt><font size=2>[root@mjs-f17-test ~]# ifconfig</font></tt>
<br><tt><font size=2>eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>
mtu 1500</font></tt>
<br><tt><font size=2> inet 10.0.0.2 netmask
255.255.255.0 broadcast 10.0.0.255</font></tt>
<br><tt><font size=2> inet6 fe80::f816:3eff:fe5f:7086
prefixlen 64 scopeid 0x20<link></font></tt>
<br><tt><font size=2> ether fa:16:3e:5f:70:86
txqueuelen 1000 (Ethernet)</font></tt>
<br><tt><font size=2> RX packets 1205699 bytes
82470224 (78.6 MiB)</font></tt>
<br><tt><font size=2> RX errors 0 dropped
3 overruns 0 frame 0</font></tt>
<br><tt><font size=2> TX packets 4382 bytes
815192 (796.0 KiB)</font></tt>
<br><tt><font size=2> TX errors 0 dropped
0 overruns 0 carrier 0 collisions 0</font></tt>
<br>
<br><tt><font size=2>lo: flags=73<UP,LOOPBACK,RUNNING> mtu
65536</font></tt>
<br><tt><font size=2> inet 127.0.0.1 netmask
255.0.0.0</font></tt>
<br><tt><font size=2> inet6 ::1 prefixlen
128 scopeid 0x10<host></font></tt>
<br><tt><font size=2> loop txqueuelen
0 (Local Loopback)</font></tt>
<br><tt><font size=2> RX packets 4 bytes
336 (336.0 B)</font></tt>
<br><tt><font size=2> RX errors 0 dropped
0 overruns 0 frame 0</font></tt>
<br><tt><font size=2> TX packets 4 bytes
336 (336.0 B)</font></tt>
<br><tt><font size=2> TX errors 0 dropped
0 overruns 0 carrier 0 collisions 0</font></tt>
<br>
<br><tt><font size=2>[root@mjs-f17-test ~]# route -n</font></tt>
<br><tt><font size=2>Kernel IP routing table</font></tt>
<br><tt><font size=2>Destination Gateway
Genmask Flags Metric Ref Use
Iface</font></tt>
<br><tt><font size=2>0.0.0.0 10.0.0.1
0.0.0.0 UG 0
0 0 eth0</font></tt>
<br><tt><font size=2>10.0.0.0 0.0.0.0
255.255.255.0 U 0
0 0 eth0</font></tt>
<br><tt><font size=2>[root@mjs-f17-test ~]# </font></tt>
<br>
<br><font size=2 face="sans-serif">Using Horizon I associate a floating
IP address, it comes up with an address like 172.24.4.7. Now it can
ping 10.0.0.2, 10.0.0.8, and 9.0.0.191 --- but it can NOT ping 9.0.0.193
nor 8.8.8.8. Inside the VM the network config looks the same.</font>
<br>
<br><font size=2 face="sans-serif">I then dissociate the floating IP address,
and the VM goes back to being able to ping anything.</font>
<br>
<br><font size=2 face="sans-serif"> </font>