Thanks for the tip, unfortunately the interfaces are already up.<div><br></div><div> - Michael<br><br><div class="gmail_quote">On Thu, Jul 12, 2012 at 10:15 PM, Jonathan Proulx <span dir="ltr"><<a href="mailto:jon@csail.mit.edu" target="_blank">jon@csail.mit.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
I've only deployed openstack for the first time a couple weeks ago,<br>
but FWIW...<br>
<br>
I had similar symptoms on my Essex test deployment (on Ubuntu 12.04)<br>
turned out my problem was taht while the br100 bridge was up and<br>
configured the underlying eth1 physical interface was down so the bits<br>
went nowhere. 'ifconfig eth1 up' fixed all, followed ofcoures by<br>
fixing in /etc/network/interfaces as well so this happens on it's own<br>
in future.<br>
<br>
-Jon<br>
<br>
On Thu, Jul 12, 2012 at 02:56:57PM +1000, Michael Chapman wrote:<br>
:Hi all, I'm hoping I could get some assistance figuring out my networking<br>
<div><div class="h5">:problems with a small Essex test cluster. I have a small Diablo cluster<br>
:running without any problems but have hit a wall in deploying Essex.<br>
:<br>
:I can launch VMs without issue and access them from the compute host, but<br>
:from there I can't access anything except the host, DNS services, and other<br>
:VMs.<br>
:<br>
:I have separate machines running keystone, glance, postgresql, rabbit-mq<br>
:and nova-api. They're all on the .os domain with 172.22.1.X IPs<br>
:<br>
:I have one machine running nova-compute, nova-network and nova-api, with a<br>
:public address 192.43.239.175 and also an IP on the 172.22.1.X subnet in<br>
:the .os domain. It has the following nova/conf:<br>
:<br>
:--dhcpbridge_flagfile=/etc/nova/nova.conf<br>
:--dhcpbridge=/usr/bin/nova-dhcpbridge<br>
:--logdir=/var/log/nova<br>
:--state_path=/var/lib/nova<br>
:--lock_path=/var/lock/nova<br>
:--force_dhcp_release<br>
:--iscsi_helper=tgtadm<br>
:--libvirt_use_virtio_for_bridges<br>
:--connection_type=libvirt<br>
:--root_helper=sudo nova-rootwrap<br>
:--verbose<br>
:--ec2_private_dns_show_ip<br>
:<br>
:--network_manager=nova.network.manager.FlatDHCPManager<br>
:--rabbit_host=os-amqp.os<br>
:--sql_connection=postgresql://[user]:[password]@os-sql.os/nova<br>
:--image_service=nova.image.glance.GlanceImageService<br>
:--glance_api_servers=os-glance.os:9292<br>
:--auth_strategy=keystone<br>
:--scheduler_driver=nova.scheduler.simple.SimpleScheduler<br>
:--keystone_ec2_url=<a href="http://os-key.os:5000/v2.0/ec2tokens" target="_blank">http://os-key.os:5000/v2.0/ec2tokens</a><br>
:<br>
:--api_paste_config=/etc/nova/api-paste.ini<br>
:<br>
:--my_ip=192.43.239.175<br>
:--flat_interface=eth0<br>
:--public_interface=eth1<br>
:--multi_host=True<br>
:--routing_source_ip=192.43.239.175<br>
:--network_host=192.43.239.175<br>
:<br>
:--dmz_cidr=$my_ip<br>
:<br>
:--ec2_host=192.43.239.175<br>
:--ec2_dmz_host=192.43.239.175<br>
:<br>
:I believe I'm seeing a natting issue of some sort - my VMs cannot ping<br>
:external IPs, though DNS seems to work.<br>
:ubuntu@monday:~$ ping <a href="http://www.google.com" target="_blank">www.google.com</a><br>
:PING <a href="http://www.l.google.com" target="_blank">www.l.google.com</a> (74.125.237.148) 56(84) bytes of data.<br>
:<AWKWARD SILENCE><br>
:<br>
:When I do a tcpdump on the compute host things seem fairly normal, even<br>
:though nothing is getting back to the VM<br>
:<br>
:root@ncios1:~# tcpdump icmp -i br100<br>
:tcpdump: verbose output suppressed, use -v or -vv for full protocol decode<br>
:listening on br100, link-type EN10MB (Ethernet), capture size 65535 bytes<br>
:14:35:28.046416 IP 10.0.0.8 > <a href="http://syd01s13-in-f20.1e100.net" target="_blank">syd01s13-in-f20.1e100.net</a>: ICMP echo request,<br>
:id 5002, seq 9, length 64<br>
:14:35:28.051477 IP <a href="http://syd01s13-in-f20.1e100.net" target="_blank">syd01s13-in-f20.1e100.net</a> > <a href="http://10.0.0.8" target="_blank">10.0.0.8</a>: ICMP echo reply,<br>
:id 5002, seq 9, length 64<br>
:14:35:29.054505 IP 10.0.0.8 > <a href="http://syd01s13-in-f20.1e100.net" target="_blank">syd01s13-in-f20.1e100.net</a>: ICMP echo request,<br>
:id 5002, seq 10, length 64<br>
:14:35:29.059556 IP <a href="http://syd01s13-in-f20.1e100.net" target="_blank">syd01s13-in-f20.1e100.net</a> > <a href="http://10.0.0.8" target="_blank">10.0.0.8</a>: ICMP echo reply,<br>
:id 5002, seq 10, length 64<br>
:<br>
:I've pored over the iptables nat rules and can't see anything amiss apart<br>
:from the masquerades that are automatically added: (I've cut out some empty<br>
:chains for brevity)<br>
:<br>
:root@ncios1:~# iptables -L -t nat -v<br>
:Chain PREROUTING (policy ACCEPT 22 packets, 2153 bytes)<br>
: pkts bytes target prot opt in out source<br>
:destination<br>
: 22 2153 nova-network-PREROUTING all -- any any anywhere<br>
: anywhere<br>
: 22 2153 nova-compute-PREROUTING all -- any any anywhere<br>
: anywhere<br>
: 22 2153 nova-api-PREROUTING all -- any any anywhere<br>
: anywhere<br>
:<br>
:Chain INPUT (policy ACCEPT 12 packets, 1573 bytes)<br>
: pkts bytes target prot opt in out source<br>
:destination<br>
:<br>
:Chain OUTPUT (policy ACCEPT 31 packets, 2021 bytes)<br>
: pkts bytes target prot opt in out source<br>
:destination<br>
: 31 2021 nova-network-OUTPUT all -- any any anywhere<br>
: anywhere<br>
: 31 2021 nova-compute-OUTPUT all -- any any anywhere<br>
: anywhere<br>
: 31 2021 nova-api-OUTPUT all -- any any anywhere<br>
:anywhere<br>
:<br>
:Chain POSTROUTING (policy ACCEPT 30 packets, 1961 bytes)<br>
: pkts bytes target prot opt in out source<br>
:destination<br>
: 31 2021 nova-network-POSTROUTING all -- any any anywhere<br>
: anywhere<br>
: 30 1961 nova-compute-POSTROUTING all -- any any anywhere<br>
: anywhere<br>
: 30 1961 nova-api-POSTROUTING all -- any any anywhere<br>
: anywhere<br>
: 30 1961 nova-postrouting-bottom all -- any any anywhere<br>
: anywhere<br>
: 0 0 MASQUERADE tcp -- any any <a href="http://192.168.122.0/24" target="_blank">192.168.122.0/24</a> !<br>
:<a href="http://192.168.122.0/24" target="_blank">192.168.122.0/24</a> masq ports: 1024-65535<br>
: 0 0 MASQUERADE udp -- any any <a href="http://192.168.122.0/24" target="_blank">192.168.122.0/24</a> !<br>
:<a href="http://192.168.122.0/24" target="_blank">192.168.122.0/24</a> masq ports: 1024-65535<br>
: 0 0 MASQUERADE all -- any any <a href="http://192.168.122.0/24" target="_blank">192.168.122.0/24</a> !<br>
</div></div>:<a href="http://192.168.122.0/24" target="_blank">192.168.122.0/24</a><br>
:<br>
:Chain nova-api-snat (1 references)<br>
<div><div class="h5">: pkts bytes target prot opt in out source<br>
:destination<br>
: 30 1961 nova-api-float-snat all -- any any anywhere<br>
: anywhere<br>
:<br>
:Chain nova-compute-snat (1 references)<br>
: pkts bytes target prot opt in out source<br>
:destination<br>
: 30 1961 nova-compute-float-snat all -- any any anywhere<br>
: anywhere<br>
:<br>
:Chain nova-network-POSTROUTING (1 references)<br>
: pkts bytes target prot opt in out source<br>
:destination<br>
: 0 0 ACCEPT all -- any any <a href="http://10.0.0.0/8" target="_blank">10.0.0.0/8</a><br>
:<a href="http://nri5.nci.org.au" target="_blank">nri5.nci.org.au</a><br>
: 0 0 ACCEPT all -- any any <a href="http://10.0.0.0/8" target="_blank">10.0.0.0/8</a><br>
:<a href="http://nri5.nci.org.au" target="_blank">nri5.nci.org.au</a><br>
: 1 60 ACCEPT all -- any any <a href="http://10.0.0.0/8" target="_blank">10.0.0.0/8</a><br>
:<a href="http://10.0.0.0/8" target="_blank">10.0.0.0/8</a> ! ctstate DNAT<br>
:<br>
:Chain nova-network-PREROUTING (1 references)<br>
: pkts bytes target prot opt in out source<br>
:destination<br>
: 0 0 DNAT tcp -- any any anywhere<br>
:169.254.169.254 tcp dpt:http to:<a href="http://192.43.239.175:8775" target="_blank">192.43.239.175:8775</a><br>
:<br>
:Chain nova-network-snat (1 references)<br>
: pkts bytes target prot opt in out source<br>
:destination<br>
: 30 1961 nova-network-float-snat all -- any any anywhere<br>
: anywhere<br>
: 0 0 SNAT all -- any any <a href="http://10.0.0.0/8" target="_blank">10.0.0.0/8</a><br>
:anywhere to:192.43.239.175<br>
:<br>
:Chain nova-postrouting-bottom (1 references)<br>
: pkts bytes target prot opt in out source<br>
:destination<br>
: 30 1961 nova-network-snat all -- any any anywhere<br>
:anywhere<br>
: 30 1961 nova-compute-snat all -- any any anywhere<br>
:anywhere<br>
: 30 1961 nova-api-snat all -- any any anywhere<br>
:anywhere<br>
:<br>
:and the ACCEPT icmp rule seems to be there in filter for the security group<br>
:as well, though it's not being triggered for some reason:<br>
:<br>
:Chain nova-compute-inst-6 (1 references)<br>
: pkts bytes target prot opt in out source<br>
:destination<br>
: 0 0 DROP all -- any any anywhere<br>
:anywhere state INVALID<br>
: 39 6545 ACCEPT all -- any any anywhere<br>
:anywhere state RELATED,ESTABLISHED<br>
: 1 60 nova-compute-provider all -- any any anywhere<br>
: anywhere<br>
: 0 0 ACCEPT udp -- any any 10.0.0.3<br>
:anywhere udp spt:bootps dpt:bootpc<br>
: 1 60 ACCEPT all -- any any <a href="http://10.0.0.0/24" target="_blank">10.0.0.0/24</a><br>
: anywhere<br>
: 0 0 ACCEPT icmp -- any any anywhere<br>
:anywhere<br>
: 0 0 ACCEPT tcp -- any any anywhere<br>
:anywhere tcp dpt:ssh<br>
: 0 0 nova-compute-sg-fallback all -- any any anywhere<br>
: anywhere<br>
:<br>
:I've tried changing the routing source IP between using the private<br>
:172.22.1.X IP and the public one but it doesn't seem to change anything. I<br>
:tried without that config option at all and also without the network host<br>
:flag and not much seems to change.<br>
:<br>
:Any help would be much appreciated.<br>
:<br>
:<br>
:<br>
:--<br>
:Michael Chapman<br>
</div></div>:*Cloud Computing Services*<br>
:ANU Supercomputer Facility<br>
<div class="im">:Room 318, Leonard Huxley Building (#56), Mills Road<br>
:The Australian National University<br>
:Canberra ACT 0200 Australia<br>
</div>:Tel: *<a href="tel:%2B61%202%206125%207106" value="+61261257106">+61 2 6125 7106</a>*<br>
:Web: <a href="http://nci.org.au" target="_blank">http://nci.org.au</a><br>
<br>
:_______________________________________________<br>
:Mailing list: <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
:Post to : <a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>
:Unsubscribe : <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
:More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
<br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><span style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px"><div><span style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px">Michael Chapman</span></div>
<div><span style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px"><i>Cloud Computing Services</i></span></div><div><span style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px">ANU Supercomputer Facility<br>
Room 318, Leonard Huxley Building (#56), Mills Road<br>The Australian National University<br>Canberra ACT 0200 Australia</span></div><div><span style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px">Tel: <i>+61 2 6125 7106</i><br>
Web: <a href="http://nci.org.au" target="_blank">http://nci.org.au</a></span></div></span><br>
</div>