[Openstack] Networking issues in Essex
Jonathan Proulx
jon at csail.mit.edu
Thu Jul 12 12:15:10 UTC 2012
I've only deployed openstack for the first time a couple weeks ago,
but FWIW...
I had similar symptoms on my Essex test deployment (on Ubuntu 12.04)
turned out my problem was taht while the br100 bridge was up and
configured the underlying eth1 physical interface was down so the bits
went nowhere. 'ifconfig eth1 up' fixed all, followed ofcoures by
fixing in /etc/network/interfaces as well so this happens on it's own
in future.
-Jon
On Thu, Jul 12, 2012 at 02:56:57PM +1000, Michael Chapman wrote:
:Hi all, I'm hoping I could get some assistance figuring out my networking
:problems with a small Essex test cluster. I have a small Diablo cluster
:running without any problems but have hit a wall in deploying Essex.
:
:I can launch VMs without issue and access them from the compute host, but
:from there I can't access anything except the host, DNS services, and other
:VMs.
:
:I have separate machines running keystone, glance, postgresql, rabbit-mq
:and nova-api. They're all on the .os domain with 172.22.1.X IPs
:
:I have one machine running nova-compute, nova-network and nova-api, with a
:public address 192.43.239.175 and also an IP on the 172.22.1.X subnet in
:the .os domain. It has the following nova/conf:
:
:--dhcpbridge_flagfile=/etc/nova/nova.conf
:--dhcpbridge=/usr/bin/nova-dhcpbridge
:--logdir=/var/log/nova
:--state_path=/var/lib/nova
:--lock_path=/var/lock/nova
:--force_dhcp_release
:--iscsi_helper=tgtadm
:--libvirt_use_virtio_for_bridges
:--connection_type=libvirt
:--root_helper=sudo nova-rootwrap
:--verbose
:--ec2_private_dns_show_ip
:
:--network_manager=nova.network.manager.FlatDHCPManager
:--rabbit_host=os-amqp.os
:--sql_connection=postgresql://[user]:[password]@os-sql.os/nova
:--image_service=nova.image.glance.GlanceImageService
:--glance_api_servers=os-glance.os:9292
:--auth_strategy=keystone
:--scheduler_driver=nova.scheduler.simple.SimpleScheduler
:--keystone_ec2_url=http://os-key.os:5000/v2.0/ec2tokens
:
:--api_paste_config=/etc/nova/api-paste.ini
:
:--my_ip=192.43.239.175
:--flat_interface=eth0
:--public_interface=eth1
:--multi_host=True
:--routing_source_ip=192.43.239.175
:--network_host=192.43.239.175
:
:--dmz_cidr=$my_ip
:
:--ec2_host=192.43.239.175
:--ec2_dmz_host=192.43.239.175
:
:I believe I'm seeing a natting issue of some sort - my VMs cannot ping
:external IPs, though DNS seems to work.
:ubuntu at monday:~$ ping www.google.com
:PING www.l.google.com (74.125.237.148) 56(84) bytes of data.
:<AWKWARD SILENCE>
:
:When I do a tcpdump on the compute host things seem fairly normal, even
:though nothing is getting back to the VM
:
:root at ncios1:~# tcpdump icmp -i br100
:tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
:listening on br100, link-type EN10MB (Ethernet), capture size 65535 bytes
:14:35:28.046416 IP 10.0.0.8 > syd01s13-in-f20.1e100.net: ICMP echo request,
:id 5002, seq 9, length 64
:14:35:28.051477 IP syd01s13-in-f20.1e100.net > 10.0.0.8: ICMP echo reply,
:id 5002, seq 9, length 64
:14:35:29.054505 IP 10.0.0.8 > syd01s13-in-f20.1e100.net: ICMP echo request,
:id 5002, seq 10, length 64
:14:35:29.059556 IP syd01s13-in-f20.1e100.net > 10.0.0.8: ICMP echo reply,
:id 5002, seq 10, length 64
:
:I've pored over the iptables nat rules and can't see anything amiss apart
:from the masquerades that are automatically added: (I've cut out some empty
:chains for brevity)
:
:root at ncios1:~# iptables -L -t nat -v
:Chain PREROUTING (policy ACCEPT 22 packets, 2153 bytes)
: pkts bytes target prot opt in out source
:destination
: 22 2153 nova-network-PREROUTING all -- any any anywhere
: anywhere
: 22 2153 nova-compute-PREROUTING all -- any any anywhere
: anywhere
: 22 2153 nova-api-PREROUTING all -- any any anywhere
: anywhere
:
:Chain INPUT (policy ACCEPT 12 packets, 1573 bytes)
: pkts bytes target prot opt in out source
:destination
:
:Chain OUTPUT (policy ACCEPT 31 packets, 2021 bytes)
: pkts bytes target prot opt in out source
:destination
: 31 2021 nova-network-OUTPUT all -- any any anywhere
: anywhere
: 31 2021 nova-compute-OUTPUT all -- any any anywhere
: anywhere
: 31 2021 nova-api-OUTPUT all -- any any anywhere
:anywhere
:
:Chain POSTROUTING (policy ACCEPT 30 packets, 1961 bytes)
: pkts bytes target prot opt in out source
:destination
: 31 2021 nova-network-POSTROUTING all -- any any anywhere
: anywhere
: 30 1961 nova-compute-POSTROUTING all -- any any anywhere
: anywhere
: 30 1961 nova-api-POSTROUTING all -- any any anywhere
: anywhere
: 30 1961 nova-postrouting-bottom all -- any any anywhere
: anywhere
: 0 0 MASQUERADE tcp -- any any 192.168.122.0/24 !
:192.168.122.0/24 masq ports: 1024-65535
: 0 0 MASQUERADE udp -- any any 192.168.122.0/24 !
:192.168.122.0/24 masq ports: 1024-65535
: 0 0 MASQUERADE all -- any any 192.168.122.0/24 !
:192.168.122.0/24
:
:Chain nova-api-snat (1 references)
: pkts bytes target prot opt in out source
:destination
: 30 1961 nova-api-float-snat all -- any any anywhere
: anywhere
:
:Chain nova-compute-snat (1 references)
: pkts bytes target prot opt in out source
:destination
: 30 1961 nova-compute-float-snat all -- any any anywhere
: anywhere
:
:Chain nova-network-POSTROUTING (1 references)
: pkts bytes target prot opt in out source
:destination
: 0 0 ACCEPT all -- any any 10.0.0.0/8
:nri5.nci.org.au
: 0 0 ACCEPT all -- any any 10.0.0.0/8
:nri5.nci.org.au
: 1 60 ACCEPT all -- any any 10.0.0.0/8
:10.0.0.0/8 ! ctstate DNAT
:
:Chain nova-network-PREROUTING (1 references)
: pkts bytes target prot opt in out source
:destination
: 0 0 DNAT tcp -- any any anywhere
:169.254.169.254 tcp dpt:http to:192.43.239.175:8775
:
:Chain nova-network-snat (1 references)
: pkts bytes target prot opt in out source
:destination
: 30 1961 nova-network-float-snat all -- any any anywhere
: anywhere
: 0 0 SNAT all -- any any 10.0.0.0/8
:anywhere to:192.43.239.175
:
:Chain nova-postrouting-bottom (1 references)
: pkts bytes target prot opt in out source
:destination
: 30 1961 nova-network-snat all -- any any anywhere
:anywhere
: 30 1961 nova-compute-snat all -- any any anywhere
:anywhere
: 30 1961 nova-api-snat all -- any any anywhere
:anywhere
:
:and the ACCEPT icmp rule seems to be there in filter for the security group
:as well, though it's not being triggered for some reason:
:
:Chain nova-compute-inst-6 (1 references)
: pkts bytes target prot opt in out source
:destination
: 0 0 DROP all -- any any anywhere
:anywhere state INVALID
: 39 6545 ACCEPT all -- any any anywhere
:anywhere state RELATED,ESTABLISHED
: 1 60 nova-compute-provider all -- any any anywhere
: anywhere
: 0 0 ACCEPT udp -- any any 10.0.0.3
:anywhere udp spt:bootps dpt:bootpc
: 1 60 ACCEPT all -- any any 10.0.0.0/24
: anywhere
: 0 0 ACCEPT icmp -- any any anywhere
:anywhere
: 0 0 ACCEPT tcp -- any any anywhere
:anywhere tcp dpt:ssh
: 0 0 nova-compute-sg-fallback all -- any any anywhere
: anywhere
:
:I've tried changing the routing source IP between using the private
:172.22.1.X IP and the public one but it doesn't seem to change anything. I
:tried without that config option at all and also without the network host
:flag and not much seems to change.
:
:Any help would be much appreciated.
:
:
:
:--
:Michael Chapman
:*Cloud Computing Services*
:ANU Supercomputer Facility
:Room 318, Leonard Huxley Building (#56), Mills Road
:The Australian National University
:Canberra ACT 0200 Australia
:Tel: *+61 2 6125 7106*
:Web: http://nci.org.au
:_______________________________________________
:Mailing list: https://launchpad.net/~openstack
:Post to : openstack at lists.launchpad.net
:Unsubscribe : https://launchpad.net/~openstack
:More help : https://help.launchpad.net/ListHelp
More information about the Openstack
mailing list