[Openstack] Limited external access from VM created by DevStack

Mike Spreitzer mspreitz at us.ibm.com
Tue Feb 11 21:23:06 UTC 2014


I am consistently suffering a network problem in simple DevStack 
installations.  Am I doing something wrong, or is this a bug, or is it to 
be expected?

I install DevStack, using a pretty basic local.conf; the only thing it 
says that is relevant to networking is setting HOST_IP to the address of 
the machine where I am installing DevStack.  Thus, it is using nova 
networking (the default), with the default address ranges.  DevStack 
completes successfully.  I edit the default security group, completely 
opening up ICMP, TCP, and UDP.  I instantiate an image.  Using Horizon I 
log into the console of that image.  From that instance I can ping 
anywhere.  Then I associate a floating IP address with that instance. 
While that floating IP is associated, I can not ping anywhere --- that 
instance can only ping the host's address and those of other VMs on the 
same host, the instance can NOT ping other hosts on the same subnet as the 
instance's host nor anything more distant.

I get this both when installing DevStack onto bare metal and when 
installing DevStack into a VM instance.  I get this when using branch 
stable/havana and when using the master branch (over the last few weeks).

Following are the details from an example in which DevStack (master 
branch) was installed onto a bare metal machine a few days ago.  Before 
installing DevStack, the host's networking config was as follows:

ubu_wa at pvespa015:~$ ifconfig
eth0      Link encap:Ethernet  HWaddr 00:21:5e:21:04:78 
          inet addr:9.0.0.191  Bcast:9.0.1.255  Mask:255.255.254.0
          inet6 addr: fe80::221:5eff:fe21:478/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:266 errors:0 dropped:1 overruns:0 frame:0
          TX packets:147 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:24924 (24.9 KB)  TX bytes:19720 (19.7 KB)

eth1      Link encap:Ethernet  HWaddr 00:21:5e:21:04:7a 
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

lo        Link encap:Local Loopback 
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:23 errors:0 dropped:0 overruns:0 frame:0
          TX packets:23 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:2841 (2.8 KB)  TX bytes:2841 (2.8 KB)

ubu_wa at pvespa015:~$ netstat -nr
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt 
Iface
0.0.0.0         9.0.0.2         0.0.0.0         UG        0 0          0 
eth0
9.0.0.0         0.0.0.0         255.255.254.0   U         0 0          0 
eth0
169.254.0.0     0.0.0.0         255.255.0.0     U         0 0          0 
eth0
ubu_wa at pvespa015:~$

Here is the local.conf that I used:

[[local|localrc]]
HOST_IP=9.0.0.191
#SERVICE_HOST=FIXME
ADMIN_PASSWORD=POK-1428
ENABLED_SERVICES+=,heat,h-api,h-api-cfn,h-api-cw,h-eng
MYSQL_PASSWORD=$ADMIN_PASSWORD
RABBIT_PASSWORD=$ADMIN_PASSWORD
SERVICE_PASSWORD=$ADMIN_PASSWORD
DEST=/opt/stack
LOGFILE=stack.sh.log
LOGDAYS=7
LOG_COLOR=False
SCREEN_LOGDIR=$DEST/logs/screen
RECLONE=yes
KEYSTONE_CATALOG_BACKEND=sql
VOLUME_GROUP="stack-volumes"
VOLUME_NAME_PREFIX="volume-"
VOLUME_BACKING_FILE_SIZE=5130M
API_RATE_LIMIT=False
IMAGE_URLS+=",
http://fedorapeople.org/groups/heat/prebuilt-jeos-images/F17-x86_64-cfntools.qcow2
"

There were no failures reported from the DevStack installation.  After 
that installation, the host's network config looked like this:

ubu_wa at pvespa015:~$ ifconfig
eth0      Link encap:Ethernet  HWaddr 00:21:5e:21:04:78 
          inet addr:9.0.0.191  Bcast:9.0.1.255  Mask:255.255.254.0
          inet6 addr: fe80::221:5eff:fe21:478/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:677321 errors:0 dropped:161 overruns:0 frame:0
          TX packets:299006 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:858856772 (858.8 MB)  TX bytes:20708280 (20.7 MB)

eth1      Link encap:Ethernet  HWaddr 00:21:5e:21:04:7a 
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

lo        Link encap:Local Loopback 
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:277765 errors:0 dropped:0 overruns:0 frame:0
          TX packets:277765 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:537924232 (537.9 MB)  TX bytes:537924232 (537.9 MB)

virbr0    Link encap:Ethernet  HWaddr 3a:a3:37:44:79:f4 
          inet addr:192.168.122.1  Bcast:192.168.122.255 
Mask:255.255.255.0
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

ubu_wa at pvespa015:~$ netstat -nr
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt 
Iface
0.0.0.0         9.0.0.2         0.0.0.0         UG        0 0          0 
eth0
9.0.0.0         0.0.0.0         255.255.254.0   U         0 0          0 
eth0
169.254.0.0     0.0.0.0         255.255.0.0     U         0 0          0 
eth0
192.168.122.0   0.0.0.0         255.255.255.0   U         0 0          0 
virbr0
ubu_wa at pvespa015:~$

I then created and tested some VM instances.  After that, the host's 
network config looked like this:

ubu_wa at pvespa015:~$ ifconfig
br100     Link encap:Ethernet  HWaddr 00:21:5e:21:04:78 
          inet addr:10.0.0.1  Bcast:10.0.0.255  Mask:255.255.255.0
          inet6 addr: fe80::fce9:c9ff:feab:ac5/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2065 errors:0 dropped:2 overruns:0 frame:0
          TX packets:213 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:123592 (123.5 KB)  TX bytes:33219 (33.2 KB)

eth0      Link encap:Ethernet  HWaddr 00:21:5e:21:04:78 
          inet6 addr: fe80::221:5eff:fe21:478/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:691620 errors:0 dropped:209 overruns:0 frame:0
          TX packets:303320 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:860180754 (860.1 MB)  TX bytes:24551439 (24.5 MB)

eth1      Link encap:Ethernet  HWaddr 00:21:5e:21:04:7a 
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

lo        Link encap:Local Loopback 
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:407649 errors:0 dropped:0 overruns:0 frame:0
          TX packets:407649 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:1043746254 (1.0 GB)  TX bytes:1043746254 (1.0 GB)

virbr0    Link encap:Ethernet  HWaddr 3a:a3:37:44:79:f4 
          inet addr:192.168.122.1  Bcast:192.168.122.255 
Mask:255.255.255.0
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

vnet0     Link encap:Ethernet  HWaddr fe:16:3e:59:bf:df 
          inet6 addr: fe80::fc16:3eff:fe59:bfdf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:75 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1843 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:7575 (7.5 KB)  TX bytes:132432 (132.4 KB)

ubu_wa at pvespa015:~$ netstat -nr
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt 
Iface
0.0.0.0         9.0.0.2         0.0.0.0         UG        0 0          0 
br100
9.0.0.0         0.0.0.0         255.255.254.0   U         0 0          0 
br100
10.0.0.0        0.0.0.0         255.255.255.0   U         0 0          0 
br100
192.168.122.0   0.0.0.0         255.255.255.0   U         0 0          0 
virbr0
ubu_wa at pvespa015:~$

Following are some examples from an instance of F17 with private IP 
address 10.0.0.2.  While it has no floating IP address, it can ping 
10.0.0.2, 10.0.0.8 (a sibling VM), 9.0.0.191 (its host), 9.0.0.193 
(another machine on the same subnet), and 8.8.8.8 (something entirely 
outside IBM's intranet).  Here is what the network config looks like 
inside the VM:

[root at mjs-f17-test ~]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.0.0.2  netmask 255.255.255.0  broadcast 10.0.0.255
        inet6 fe80::f816:3eff:fe5f:7086  prefixlen 64  scopeid 0x20<link>
        ether fa:16:3e:5f:70:86  txqueuelen 1000  (Ethernet)
        RX packets 1205699  bytes 82470224 (78.6 MiB)
        RX errors 0  dropped 3  overruns 0  frame 0
        TX packets 4382  bytes 815192 (796.0 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 4  bytes 336 (336.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 4  bytes 336 (336.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root at mjs-f17-test ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use 
Iface
0.0.0.0         10.0.0.1        0.0.0.0         UG    0      0        0 
eth0
10.0.0.0        0.0.0.0         255.255.255.0   U     0      0        0 
eth0
[root at mjs-f17-test ~]# 

Using Horizon I associate a floating IP address, it comes up with an 
address like 172.24.4.7.  Now it can ping 10.0.0.2, 10.0.0.8, and 
9.0.0.191 --- but it can NOT ping 9.0.0.193 nor 8.8.8.8.  Inside the VM 
the network config looks the same.

I then dissociate the floating IP address, and the VM goes back to being 
able to ping anything.

 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20140211/0ed7b257/attachment.html>


More information about the Openstack mailing list