Open Stack

Hi Derek,

Moments after I hit "send" I figured it out.  This is a bug in libvirt
and has been addressed in Fedora and libvirt, but not in RHELv6.3, yet.

When the libvirt "default" network is started it can mess up the routing
table on the system.  Removing the virbr0 bridge device and removing the
default.xml file from /var/lib/libvirt/network and
/etc/libvirt/qemu/network/autostart resolves the issue and I can ping
and ssh into all the VMs, now.

Also, recall that the VMs don't actually get a public, floating IP when
they are assigned one - only iptable rules are constructed to forward
public traffic from the public NIC to the internal VM private IP, so I
wouldn't actually see the public IP on any of the VMs.

Cheers!
Dan


On 09/18/2012 07:56 AM, Derek Higgins wrote:
> Hi Dan,
> 
>    Are you able to verify the VM's are getting the ipaddress you expect
> by using novnc or virt-manager?
> 
> Derek.
> 
> On 09/17/2012 10:16 PM, yocum at redhat.com wrote:
>> Hi all,
>>
>> I've run into some networking problems and I don't know where to go from
>> here.
>>
>> On one host machine, I can ssh into the all (2) floating public IPs that
>> the VMs have been assigned.
>>
>> On one host machine, I can't ssh or ping any (again, 2) of the public
>> IPs that the VMs have been assigned.
>>
>> On yet another host machine, I can ssh into 1 VM but not the other 2.
>>
>> I'm using flatDHCP with multi-host=T.  The diagram of my layout looks
>> very much like this diagram:
>>
>> http://unchainyourbrain.com/images/stories/programming/ha-net.jpg
>>
>> which comes from this article:
>>
>> http://unchainyourbrain.com/openstack/13-networking-in-nova
>>
>> Here's my nova.conf from my controller node (RHEL 6.3):
>>
>> bindir=/usr/bin
>> connection_type=libvirt
>> debug=true
>> default_project=osop
>> firewall_driver=nova.virt.libvirt.firewall.IptablesFirewallDriver
>> glance_api_servers=my-controller.some.tld.com:9292
>> lock_path=/var/lib/nova/tmp
>> logdir=/var/log/nova
>> my_ip = 10.4.59.6
>> network_manager=nova.network.manager.FlatDHCPManager
>> root_helper=sudo nova-rootwrap
>> state_path=/var/lib/nova
>> sql_connection=mysql://nova:nova@my-controller.some.tld.com/nova
>> rpc_backend=nova.rpc.impl_qpid
>> qpid_hostname=my-controller.some.tld.com
>> qpid_port=5672
>> quota_floating_ips=120
>> quota_instances=120
>> quota_ram=151200
>> auth_strategy=keystone
>> vpn_client_template=/usr/share/nova/client.ovpn.template
>> credentials_template=/usr/share/nova/novarc.template
>> novncproxy_base_url=http://my-controller.some.tld.com:6080/vnc_auto.html
>> novncproxy_host = 0.0.0.0
>> novncproxy_port = 6080
>> vnc_enabled=true
>> vncserver_listen=127.0.0.1
>> vncserver_proxyclient_address=127.0.0.1
>> dhcpbridge=$bindir/nova-dhcpbridge
>> dhcpbridge_flagfile=/etc/nova/nova.conf
>> public_interface=eth4
>> auto_assign_floating_ip=true
>> default_floating_pool=osop-pub
>> dhcp_domain=osop-local
>> fixed_range=192.168.59.0/24
>> flat_interface=eth5
>> flat_network_bridge=osop-virtbr
>> flat_network_dns=8.8.4.4
>> floating_range=209.123.123.0/24
>> force_dhcp_release=true
>> multi_host=true
>> network_host=10.4.59.6
>> network_size=256
>> num_networks=1
>> injected_network_template=/usr/share/nova/interfaces.template
>> libvirt_inject_partition=-1
>> libvirt_nonblocking=true
>> libvirt_type=kvm
>> libvirt_xml_template=/usr/share/nova/libvirt.xml.template
>> remove_unused_base_images=true
>> iscsi_helper=tgtadm
>> iscsi_ip_address=my-controller.some.tld.com
>>
>> routing tables, iptable rules, and arp caches all look essentially the
>> same for all the systems and yet I can't ssh into some VMs.
>>
>> I'm at a loss.  I hope someone can help.
>>
>> Thanks,
>> Dan
>>
> 

-- 
Dan Yocum
Sr. Systems Engineer
OpenShift | PaaS by Red Hat
dyocum at redhat.com