<html><head><style>.em-message p{margin:0px}</style></head><body><div class="em-message"><font face="Arial, Helvetica, sans-serif"><span style="font-size: 12px;">Hi guys,</span></font>
<div style="color: rgb(0, 0, 0); font-family: Arial, Helvetica, sans-serif; font-size: 12px; "><br></div><div style="color: rgb(0, 0, 0); font-family: Arial, Helvetica, sans-serif; font-size: 12px; ">I am trying to set up an openstack small cluster and I am facing a problem.</div><div style="color: rgb(0, 0, 0); font-family: Arial, Helvetica, sans-serif; font-size: 12px; ">I have two nodes, a controller node on which I have installed all the nova-* services and a compute node on which I have installed nova-compute and nova-network.</div><div style="color: rgb(0, 0, 0); font-family: Arial, Helvetica, sans-serif; font-size: 12px; "><br></div><div style="color: rgb(0, 0, 0); font-family: Arial, Helvetica, sans-serif; font-size: 12px; ">When I run 'nova-manage service list', I only see the nova-compute service from the compute-node, not the other services from the controller node. However, if I run 'ps aux | grep nova-' on the controller node, all the nova-* are up and running. It seems that my controller's nova-* services do not want to register to the mysql DB.</div><div style="color: rgb(0, 0, 0); font-family: Arial, Helvetica, sans-serif; font-size: 12px; "><br></div><div style="color: rgb(0, 0, 0); "><font size="2" face="arial"><div>My guess is that services are not able to connect to the local mysql db on the controller node may be due to iptables bad settings but I am not able to figure it out.</div><div>Worth to mention it, my nova-controller node is running in a virtual machine on virtual box, both nodes are ubuntu natty server amd64.</div><div><br></div><div>If someone have any idea on that problem, please let me know, I added some more information about my configuration in attachment below.</div><div><br></div><div>Than you in advance.</div><div><br></div><div>Best Regards,</div><div>Ivan Frain</div></font></div><div style="color: rgb(0, 0, 0); font-family: Arial, Helvetica, sans-serif; font-size: 12px; "><br></div><div style="color: rgb(0, 0, 0); font-family: Arial, Helvetica, sans-serif; font-size: 12px; "><br></div><div style="color: rgb(0, 0, 0); font-family: Arial, Helvetica, sans-serif; font-size: 12px; "><b>Here is my nova.conf file: it is the same on both machines: </b></div><blockquote class="webkit-indent-blockquote" style="color: rgb(0, 0, 0); font-size: 12px; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 40px; border-top-style: none; border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; "><div><div><font face="'courier new'">--dhcpbridge_flagfile=/etc/nova/nova.conf</font></div></div><div><div><font face="'courier new'">--dhcpbridge=/usr/bin/nova-dhcpbridge</font></div></div><div><div><font face="'courier new'">--logdir=/var/log/nova</font></div></div><div><div><font face="'courier new'">--state_path=/var/lib/nova</font></div></div><div><div><font face="'courier new'">--lock_path=/var/lock/nova</font></div></div><div><div><font face="'courier new'">--verbose</font></div></div><div><div><font face="'courier new'">--sql_connection=mysql://nova:notnova@nova-controller/nova</font></div></div><div><div><font face="'courier new'">--s3_host=nova-controller</font></div></div><div><div><font face="'courier new'">--rabbit_host=nova-controller</font></div></div><div><div><font face="'courier new'">--ec2_api=nova-controller</font></div></div><div><div><font face="'courier new'">--ec2_url=http://nova-controller:8773/services/Cloud</font></div></div><div><div><font face="'courier new'">--fixed_range=192.168.0.0/16</font></div></div><div><div><font face="'courier new'">--network_size=8</font></div></div><div><div><font face="'courier new'">--routing_source_ip=nova-controller</font></div></div><div style="font-family: Arial, Helvetica, sans-serif; "><br></div></blockquote><font face="Arial, Helvetica, sans-serif"><span style="font-size: 12px;"><b>the output of ps aux | grep nova command:</b></span></font><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"><div><font face="'courier new'" size="2">root@nova-controller:~# ps aux | grep nova</font></div></blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"><div><font face="'courier new'" size="2">rabbitmq 1118 0.9 5.3 53644 27224 ? Ssl 21:06 0:47 /usr/lib/erlang/erts-5.7.4/bin/beam -W w -K true -A30 -P 1048576 -- -root /usr/lib/erlang -progname erl -- -home /var/lib/rabbitmq -- -noshell -noinput -sname rabbit@nova-controller -boot /var/lib/rabbitmq/mnesia/rabbit@nova-controller-plugins-expand/rabbit -kernel inet_default_listen_options [{nodelay,true}] -kernel inet_default_connect_options [{nodelay,true}] -sasl errlog_type error -kernel error_logger {file,"/var/log/rabbitmq/rabbit@nova-controller.log"} -sasl sasl_error_logger {file,"/var/log/rabbitmq/rabbit@nova-controller-sasl.log"} -os_mon start_cpu_sup true -os_mon start_disksup false -os_mon start_memsup false -mnesia dir "/var/lib/rabbitmq/mnesia/rabbit@nova-controller" -noshell -noinput</font></div></blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"><div><font face="'courier new'" size="2">root 19856 0.1 3.2 61228 16232 ? S 22:24 0:00 /usr/bin/python /usr/bin/nova-objectstore --uid 108 --gid 65534 --pidfile /var/run/nova/nova-objectstore.pid --flagfile=/etc/nova/nova.conf --nodaemon --logfile=/var/log/nova/nova-objectstore.log</font></div></blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"><div><font face="'courier new'" size="2">nova 23074 0.0 0.2 35708 1280 ? Ss 22:26 0:00 su -c nova-compute --flagfile=/etc/nova/nova.conf nova</font></div></blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"><div><font face="'courier new'" size="2">nova 23075 30.3 8.2 145228 41740 ? R 22:26 0:00 /usr/bin/python /usr/bin/nova-compute --flagfile=/etc/nova/nova.conf</font></div></blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"><div><font face="'courier new'" size="2">nova 23132 0.0 0.2 35708 1276 ? Ss 22:26 0:00 su -c nova-api --flagfile=/etc/nova/nova.conf nova</font></div></blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"><div><font face="'courier new'" size="2">nova 23133 35.0 5.3 89264 26884 ? R 22:26 0:00 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf</font></div></blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"><div><font face="'courier new'" size="2">nova 23145 0.0 0.2 35708 1280 ? Ss 22:26 0:00 su -c nova-network --flagfile=/etc/nova/nova.conf nova</font></div></blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"><div><font face="'courier new'" size="2">nova 23146 23.0 4.2 75476 21596 ? R 22:26 0:00 /usr/bin/python /usr/bin/nova-network --flagfile=/etc/nova/nova.conf</font></div></blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"><div><font face="'courier new'" size="2">nova 23152 0.0 0.2 35708 1280 ? Ss 22:26 0:00 su -c nova-scheduler --flagfile=/etc/nova/nova.conf nova</font></div></blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"><div><font face="'courier new'" size="2">nova 23153 0.0 3.7 72924 19112 ? R 22:26 0:00 /usr/bin/python /usr/bin/nova-scheduler --flagfile=/etc/nova/nova.conf</font></div></blockquote><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"><div><font face="'courier new'" size="2">root 23167 0.0 0.2 9136 1060 pts/1 S+ 22:26 0:00 grep --color=auto nova</font></div></blockquote></blockquote><div><br></div><div><b>The output of service-manage command:</b></div><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"><div><div><font face="'courier new'" size="2">root@nova-controller:~# nova-manage service list</font></div><div><font face="'courier new'" size="2">Obelix nova-compute enabled :-) 2011-08-08 20:27:26</font></div></div></blockquote><br><div>NB: Obelix is the name of my compute node.</div><div><br></div><div><br></div><div><div><b>mysql> select Host,User,Password from user;</b></div><div>+-----------------+------------------+-------------------------------------------+</div><div>| Host | User | Password |</div><div>+-----------------+------------------+-------------------------------------------+</div><div>| localhost | root | *0BE3B501084D35F4C66DD3AC4569EAE5EA738212 |</div><div>| nova-controller | root | *0BE3B501084D35F4C66DD3AC4569EAE5EA738212 |</div><div>| 127.0.0.1 | root | *0BE3B501084D35F4C66DD3AC4569EAE5EA738212 |</div><div>| localhost | debian-sys-maint | *B23864E8C293F7A6E977A8CF31AC4ED99570A161 |</div><div>| % | nova | *8134721C1A2B651300B26AC135712CD373E4D471 |</div><div>+-----------------+------------------+-------------------------------------------+</div></div><div><br></div><div><b>A problem in the output from nova-network log (iptables problem):</b></div><div><div>root@nova-controller:~# tail -n 50 /var/log/nova/nova-network.log </div><div>2011-08-08 19:34:11,852 DEBUG nova.utils [-] Attempting to grab file lock "iptables" for method "apply"... from (pid=10073) inner /usr/lib/pymodules/python2.7/nova/utils.py:599</div><div>2011-08-08 19:34:11,854 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-save -t filter from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:150</div><div>2011-08-08 19:34:11,870 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-restore from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:150</div><div>2011-08-08 19:34:11,911 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-save -t nat from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:150</div><div>2011-08-08 19:34:11,929 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-restore from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:150</div><div>2011-08-08 19:34:11,963 DEBUG nova.utils [-] Result was 2 from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:166</div><div>2011-08-08 19:34:11,963 DEBUG nova.utils [-] ['sudo', 'iptables-restore'] failed. Retrying. from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:179</div><div>2011-08-08 19:34:12,776 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-restore from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:150</div><div>2011-08-08 19:34:12,802 DEBUG nova.utils [-] Result was 2 from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:166</div><div>2011-08-08 19:34:12,803 DEBUG nova.utils [-] ['sudo', 'iptables-restore'] failed. Retrying. from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:179</div><div>2011-08-08 19:34:14,616 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-restore from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:150</div><div>2011-08-08 19:34:14,641 DEBUG nova.utils [-] Result was 2 from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:166</div><div>2011-08-08 19:34:14,642 DEBUG nova.utils [-] ['sudo', 'iptables-restore'] failed. Retrying. from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:179</div><div>2011-08-08 19:34:15,723 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-restore from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:150</div><div>2011-08-08 19:34:15,742 DEBUG nova.utils [-] Result was 2 from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:166</div><div>2011-08-08 19:34:15,742 DEBUG nova.utils [-] ['sudo', 'iptables-restore'] failed. Retrying. from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:179</div><div>2011-08-08 19:34:16,703 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-restore from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:150</div><div>2011-08-08 19:34:16,732 DEBUG nova.utils [-] Result was 2 from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:166</div><div>2011-08-08 19:34:16,732 CRITICAL nova [-] Unexpected error while running command.</div><div>Command: sudo iptables-restore</div><div>Exit code: 2</div><div>Stdout: ''</div><div>Stderr: 'iptables-restore v1.4.10: Bad IP address "nova"\n\nError occurred at line: 18\nTry `iptables-restore -h\' or \'iptables-restore --help\' for more information.\n'</div><div>(nova): TRACE: Traceback (most recent call last):</div><div>(nova): TRACE: File "/usr/bin/nova-network", line 48, in <module></div><div>(nova): TRACE: service.serve()</div><div>(nova): TRACE: File "/usr/lib/pymodules/python2.7/nova/service.py", line 290, in serve</div><div>(nova): TRACE: x.start()</div><div>(nova): TRACE: File "/usr/lib/pymodules/python2.7/nova/service.py", line 84, in start</div><div>(nova): TRACE: self.manager.init_host()</div><div>(nova): TRACE: File "/usr/lib/pymodules/python2.7/nova/network/manager.py", line 499, in init_host</div><div>(nova): TRACE: super(VlanManager, self).init_host()</div><div>(nova): TRACE: File "/usr/lib/pymodules/python2.7/nova/network/manager.py", line 128, in init_host</div><div>(nova): TRACE: self.driver.init_host()</div><div>(nova): TRACE: File "/usr/lib/pymodules/python2.7/nova/network/linux_net.py", line 378, in init_host</div><div>(nova): TRACE: iptables_manager.apply()</div><div>(nova): TRACE: File "/usr/lib/pymodules/python2.7/nova/utils.py", line 607, in inner</div><div>(nova): TRACE: retval = f(*args, **kwargs)</div><div>(nova): TRACE: File "/usr/lib/pymodules/python2.7/nova/network/linux_net.py", line 292, in apply</div><div>(nova): TRACE: attempts=5)</div><div>(nova): TRACE: File "/usr/lib/pymodules/python2.7/nova/network/linux_net.py", line 666, in _execute</div><div>(nova): TRACE: return utils.execute(*cmd, **kwargs)</div><div>(nova): TRACE: File "/usr/lib/pymodules/python2.7/nova/utils.py", line 173, in execute</div><div>(nova): TRACE: cmd=' '.join(cmd))</div><div>(nova): TRACE: ProcessExecutionError: Unexpected error while running command.</div><div>(nova): TRACE: Command: sudo iptables-restore</div><div>(nova): TRACE: Exit code: 2</div><div>(nova): TRACE: Stdout: ''</div><div>(nova): TRACE: Stderr: 'iptables-restore v1.4.10: Bad IP address "nova"\n\nError occurred at line: 18\nTry `iptables-restore -h\' or \'iptables-restore --help\' for more information.\n'</div><div>(nova): TRACE: </div></div><div><br></div><div><b>A problem of db connection in nova compute log from the controller:</b></div><div>(nova): TRACE: ProgrammingError: (ProgrammingError) (1146, "Table 'nova.instances' doesn't exist") 'SELECT instances.created_at AS instances_created_at, </div><div><br></div><div>[......]</div><div><div><br></div><div>ting_ips_1.deleted = %s \nWHERE instances.host = %s AND instances.deleted = %s' (False, False, False, False, False, 'nova-controller', False)</div><div>(nova): TRACE: </div></div><div><br></div><div><b>ifconfig and iptables of the controller node:</b></div><div><div>root@nova-controller:~# ifconfig</div><div>br100 Link encap:Ethernet HWaddr 08:00:27:3d:e9:7f </div><div> inet addr:192.168.1.4 Bcast:192.168.1.255 Mask:255.255.255.0</div><div> inet6 addr: fe80::a00:27ff:fe3d:e97f/64 Scope:Link</div><div> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1</div><div> RX packets:352069 errors:0 dropped:0 overruns:0 frame:0</div><div> TX packets:178694 errors:0 dropped:0 overruns:0 carrier:0</div><div> collisions:0 txqueuelen:0 </div><div> RX bytes:28108787 (28.1 MB) TX bytes:18241764 (18.2 MB)</div><div><br></div><div>eth0 Link encap:Ethernet HWaddr 08:00:27:3d:e9:7f </div><div> inet6 addr: fe80::a00:27ff:fe3d:e97f/64 Scope:Link</div><div> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1</div><div> RX packets:352848 errors:0 dropped:0 overruns:0 frame:0</div><div> TX packets:179157 errors:0 dropped:0 overruns:0 carrier:0</div><div> collisions:0 txqueuelen:1000 </div><div> RX bytes:33117855 (33.1 MB) TX bytes:18271426 (18.2 MB)</div><div><br></div><div>lo Link encap:Local Loopback </div><div> inet addr:127.0.0.1 Mask:255.0.0.0</div><div> inet6 addr: ::1/128 Scope:Host</div><div> UP LOOPBACK RUNNING MTU:16436 Metric:1</div><div> RX packets:498 errors:0 dropped:0 overruns:0 frame:0</div><div> TX packets:498 errors:0 dropped:0 overruns:0 carrier:0</div><div> collisions:0 txqueuelen:0 </div><div> RX bytes:108091 (108.0 KB) TX bytes:108091 (108.0 KB)</div><div><br></div><div>virbr0 Link encap:Ethernet HWaddr 22:0f:89:53:eb:82 </div><div> inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0</div><div> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1</div><div> RX packets:0 errors:0 dropped:0 overruns:0 frame:0</div><div> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0</div><div> collisions:0 txqueuelen:0 </div><div> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)</div><div><br></div><div>root@nova-controller:~# iptables --list</div><div>Chain INPUT (policy ACCEPT)</div><div>target prot opt source destination </div><div>ACCEPT udp -- anywhere anywhere udp dpt:domain </div><div>ACCEPT tcp -- anywhere anywhere tcp dpt:domain </div><div>ACCEPT udp -- anywhere anywhere udp dpt:bootps </div><div>ACCEPT tcp -- anywhere anywhere tcp dpt:bootps </div><div><br></div><div>Chain FORWARD (policy ACCEPT)</div><div>target prot opt source destination </div><div>ACCEPT all -- anywhere 192.168.122.0/24 state RELATED,ESTABLISHED </div><div>ACCEPT all -- 192.168.122.0/24 anywhere </div><div>ACCEPT all -- anywhere anywhere </div><div>REJECT all -- anywhere anywhere reject-with icmp-port-unreachable </div><div>REJECT all -- anywhere anywhere reject-with icmp-port-unreachable </div><div><br></div><div>Chain OUTPUT (policy ACCEPT)</div><div>target prot opt source destination </div></div><div><br></div></div></body></html>