<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
I'm setting up a modest cloud for internal use. Currently I have the following configuration:
<div><br>
</div>
<div>Server1 (nova) is running all nova services EXCEPT nova compute.</div>
<div>Server 2 (nova1) is running only nova-compute.</div>
<div>Servers3-12 are idle, but will eventually run nova-compute</div>
<div><br>
</div>
<div>Server 1 is configured with public_interface eth0 and vlan_interface eth1. The nova.conf file is as follows</div>
<div><br>
</div>
<div>
<div><font class="Apple-style-span" face="Courier">--dhcpbridge_flagfile=/etc/nova/nova.conf</font></div>
<div><font class="Apple-style-span" face="Courier">--dhcpbridge=/usr/bin/nova-dhcpbridge</font></div>
<div><font class="Apple-style-span" face="Courier">--logdir=/var/log/nova</font></div>
<div><font class="Apple-style-span" face="Courier">--state_path=/var/lib/nova</font></div>
<div><font class="Apple-style-span" face="Courier">--lock_path=/var/lock/nova</font></div>
<div><font class="Apple-style-span" face="Courier">--verbose</font></div>
<div><font class="Apple-style-span" face="Courier">--s3_host=173.23.181.1</font></div>
<div><font class="Apple-style-span" face="Courier">--rabbit_host=173.23.181.1</font></div>
<div><font class="Apple-style-span" face="Courier">--cc_host=173.23.181.1</font></div>
<div><font class="Apple-style-span" face="Courier">--ec2_url=<a href="http://173.23.181.1:8773/services/Cloud">http://173.23.181.1:8773/services/Cloud</a></font></div>
<div><font class="Apple-style-span" face="Courier">--nova_url=<a href="http://173.23.181.1:8774/v1.1/">http://173.23.181.1:8774/v1.1/</a></font></div>
<div><font class="Apple-style-span" face="Courier">--fixed_range=192.168.64.0/18</font></div>
<div><font class="Apple-style-span" face="Courier">--network_size=8</font></div>
<div><font class="Apple-style-span" face="Courier">--FAKE_subdomain=cloud</font></div>
<div><font class="Apple-style-span" face="Courier">--routing_source_ip=173.23.181.1</font></div>
<div><font class="Apple-style-span" face="Courier">--sql_connection=<a href="mysql://nova:nova@173.23.181.1/nova">mysql://nova:nova@173.23.181.1/nova</a></font></div>
<div><font class="Apple-style-span" face="Courier">--glance_api_servers=192.168.10.1:9292</font></div>
<div><font class="Apple-style-span" face="Courier">--public_interface=eth0</font></div>
<div><font class="Apple-style-span" face="Courier">--vlan_interface=eth1</font></div>
<div><font class="Apple-style-span" face="Courier">--use_deprecated_auth</font></div>
<div><font class="Apple-style-span" face="Courier">--iscsi_ip_prefix=192.168.10.</font></div>
</div>
<div><br>
</div>
<div>Server 2 is configured with vlan_interface eth3. The nova.conf file is practically identical to Server1 and is as follows</div>
<div><br>
</div>
<div>
<div><font class="Apple-style-span" face="Courier">--dhcpbridge_flagfile=/etc/nova/nova.conf</font></div>
<div><font class="Apple-style-span" face="Courier">--dhcpbridge=/usr/bin/nova-dhcpbridge</font></div>
<div><font class="Apple-style-span" face="Courier">--logdir=/var/log/nova</font></div>
<div><font class="Apple-style-span" face="Courier">--state_path=/var/lib/nova</font></div>
<div><font class="Apple-style-span" face="Courier">--lock_path=/var/lock/nova</font></div>
<div><font class="Apple-style-span" face="Courier">--verbose</font></div>
<div><font class="Apple-style-span" face="Courier">--s3_host=173.23.181.1</font></div>
<div><font class="Apple-style-span" face="Courier">--rabbit_host=173.23.181.1</font></div>
<div><font class="Apple-style-span" face="Courier">--cc_host=173.23.181.1</font></div>
<div><font class="Apple-style-span" face="Courier">--ec2_url=<a href="http://173.23.181.1:8773/services/Cloud">http://173.23.181.1:8773/services/Cloud</a></font></div>
<div><font class="Apple-style-span" face="Courier">--nova_url=<a href="http://173.23.181.1:8774/v1.1/">http://173.23.181.1:8774/v1.1/</a></font></div>
<div><font class="Apple-style-span" face="Courier">--fixed_range=192.168.64.0/18</font></div>
<div><font class="Apple-style-span" face="Courier">--network_size=8</font></div>
<div><font class="Apple-style-span" face="Courier">--FAKE_subdomain=cloud</font></div>
<div><font class="Apple-style-span" face="Courier">--routing_source_ip=173.23.181.1</font></div>
<div><font class="Apple-style-span" face="Courier">--sql_connection=<a href="mysql://nova:nova@173.23.181.1/nova">mysql://nova:nova@173.23.181.1/nova</a></font></div>
<div><font class="Apple-style-span" face="Courier">--glance_api_servers=192.168.10.1:9292</font></div>
<div><font class="Apple-style-span" face="Courier">--vlan_interface=eth3</font></div>
<div><font class="Apple-style-span" face="Courier">--use_deprecated_auth</font></div>
<div><font class="Apple-style-span" face="Courier">--iscsi_ip_prefix=192.168.10.</font></div>
</div>
<div><br>
</div>
<div>I've setup 1 private network 192.168.64.0/19.</div>
<div><br>
</div>
<div>I'm seeing the following behavior and wonder if this is a bug. When starting an instance, if the bridge interface does not exist on Server2 (br100), the instance fails to start and the nova-compute.log file shows the following error:</div>
<div><br>
</div>
<div>
<div><font class="Apple-style-span" face="Courier">2012-01-25 15:59:26,644 ERROR nova.exception [-] Uncaught exception</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: Traceback (most recent call last):</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 98, in w</font></div>
<div><font class="Apple-style-span" face="Courier">rapped</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: return f(*args, **kw)</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py",</font></div>
<div><font class="Apple-style-span" face="Courier"> line 672, in spawn</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: block_device_info=block_device_info)</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py",</font></div>
<div><font class="Apple-style-span" face="Courier"> line 1272, in to_xml</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: block_device_info)</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py",</font></div>
<div><font class="Apple-style-span" face="Courier"> line 1170, in _prepare_xml_info</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: nics.append(self.vif_driver.plug(instance, network, mapping))</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/vif.py", line 8</font></div>
<div><font class="Apple-style-span" face="Courier">6, in plug</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: network['bridge_interface'])</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: File "/usr/lib/python2.7/dist-packages/nova/network/linux_net.py", line </font></div>
<div><font class="Apple-style-span" face="Courier">889, in ensure_vlan_bridge</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: bridge_interface, mac_address)</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 688, in inne</font></div>
<div><font class="Apple-style-span" face="Courier">r</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: retval = f(*args, **kwargs)</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: File "/usr/lib/python2.7/dist-packages/nova/network/linux_net.py", line </font></div>
<div><font class="Apple-style-span" face="Courier">909, in ensure_vlan</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: _execute('ip', 'link', 'set', interface, 'up', run_as_root=True)</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: File "/usr/lib/python2.7/dist-packages/nova/network/linux_net.py", line </font></div>
<div><font class="Apple-style-span" face="Courier">745, in _execute</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: return utils.execute(*cmd, **kwargs)</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 191, in exec</font></div>
<div><font class="Apple-style-span" face="Courier">ute</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: cmd=' '.join(cmd))</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: ProcessExecutionError: Unexpected error while running command.</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: Command: sudo ip link set vlan100 up</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: Exit code: 2</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: Stdout: ''</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: Stderr: 'RTNETLINK answers: Network is down\n'</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.exception): TRACE: </font></div>
<div><font class="Apple-style-span" face="Courier">2012-01-25 15:59:26,645 ERROR nova.compute.manager [-] Instance '11' failed to spawn. Is virtualiza</font></div>
<div><font class="Apple-style-span" face="Courier">tion enabled in the BIOS? Details: Unexpected error while running command.</font></div>
<div><font class="Apple-style-span" face="Courier">Command: sudo ip link set vlan100 up</font></div>
<div><font class="Apple-style-span" face="Courier">Exit code: 2</font></div>
<div><font class="Apple-style-span" face="Courier">Stdout: ''</font></div>
<div><font class="Apple-style-span" face="Courier">Stderr: 'RTNETLINK answers: Network is down\n'</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.compute.manager): TRACE: Traceback (most recent call last):</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.compute.manager): TRACE: File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", l</font></div>
<div><font class="Apple-style-span" face="Courier">ine 424, in _run_instance</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.compute.manager): TRACE: network_info, block_device_info)</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.compute.manager): TRACE: File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 12</font></div>
<div><font class="Apple-style-span" face="Courier">9, in wrapped</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.compute.manager): TRACE: raise Error(str(e))</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.compute.manager): TRACE: Error: Unexpected error while running command.</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.compute.manager): TRACE: Command: sudo ip link set vlan100 up</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.compute.manager): TRACE: Exit code: 2</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.compute.manager): TRACE: Stdout: ''</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.compute.manager): TRACE: Stderr: 'RTNETLINK answers: Network is down\n'</font></div>
<div><font class="Apple-style-span" face="Courier">(nova.compute.manager): TRACE: </font></div>
</div>
<div><br>
</div>
<div>if I kill the pending instance on the controller node and re-issue the command to start and instance, I see the following VLAN configuration (vlan100)</div>
<div><br>
</div>
<div>
<div><font class="Apple-style-span" face="Courier">vlan100 VID: 100<span class="Apple-tab-span" style="white-space:pre">
</span>REORDER_HDR: 1 dev->priv_flags: 4001</font></div>
<div><font class="Apple-style-span" face="Courier"> total frames received 0</font></div>
<div><font class="Apple-style-span" face="Courier"> total bytes received 0</font></div>
<div><font class="Apple-style-span" face="Courier"> Broadcast/Multicast Rcvd 0</font></div>
<div><font class="Apple-style-span" face="Courier"><br>
</font></div>
<div><font class="Apple-style-span" face="Courier"> total frames transmitted 0</font></div>
<div><font class="Apple-style-span" face="Courier"> total bytes transmitted 0</font></div>
<div><font class="Apple-style-span" face="Courier">Device: eth1</font></div>
<div><font class="Apple-style-span" face="Courier">INGRESS priority mappings: 0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0</font></div>
<div><font class="Apple-style-span" face="Courier"> EGRESS priority mappings: </font></div>
</div>
<div><br>
</div>
<div>Note that the VLAN is using eth1 when (on the compute node) it should be using eth3.</div>
<div><br>
</div>
<div>What am I missing?</div>
<div><br>
</div>
<div>Regards,</div>
<div>Ross</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
</body>
</html>