<br><br><div class="gmail_quote">2012/12/10 Vishvananda Ishaya <span dir="ltr"><<a href="mailto:vishvananda@gmail.com" target="_blank">vishvananda@gmail.com</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">Check rabbitmqctl list_queues and see if there are queues that have nonzero entries. That means messages are being sent but not picked up by the nodes.</div></blockquote><div><br></div><div>
Here is the output of the command -- there some non zero values for glance info. Is there any way to clear this queue? </div><div><br></div><div><div>rabbitmqctl list_queues</div><div>Listing queues ...</div><div>cinder-volume<span class="Apple-tab-span" style="white-space:pre"> </span>0</div>
<div>network.computeDomU07<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>compute.computeDomU07<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>network.computeDomU05<span class="Apple-tab-span" style="white-space:pre"> </span>0</div>
<div>compute<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>scheduler.computeDomU05<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>consoleauth.computeDomU08<span class="Apple-tab-span" style="white-space:pre"> </span>0</div>
<div>network.controller<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>cinder-scheduler<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>compute_fanout_470aa521abf344f398dc98bf2f99b6e1<span class="Apple-tab-span" style="white-space:pre"> </span>0</div>
<div>scheduler.computeDomU07<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>compute.controller<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>network_fanout_209e3038f3af49849c10263ab1012e37<span class="Apple-tab-span" style="white-space:pre"> </span>0</div>
<div>cert.computeDomU07<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>scheduler.computeDomU08<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>consoleauth.computeDomU05<span class="Apple-tab-span" style="white-space:pre"> </span>0</div>
<div>glance_notifications.warn<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>consoleauth_fanout_9f3c92a646e74f9c88cb1317379d0a6a<span class="Apple-tab-span" style="white-space:pre"> </span>0</div>
<div>scheduler.controller<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>network_fanout_9cf790a90c2f49c781a8046d0ea91d30<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>cinder-volume_fanout_693d1211a2a4478b828acb91099a228b<span class="Apple-tab-span" style="white-space:pre"> </span>0</div>
<div>cert_fanout_d669d273749c432d999cdfd0a7a715a9<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>network_fanout_e4cc6507e07747d093ecc59502d62018<span class="Apple-tab-span" style="white-space:pre"> </span>0</div>
<div>consoleauth.computeDomU07<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>cert_fanout_a9734647226c4ebda2e112ece8f6f168<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>compute.computeDomU08<span class="Apple-tab-span" style="white-space:pre"> </span>0</div>
<div>cert.computeDomU05<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>scheduler_fanout_6c29f818954243468653858252666f2a<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>cert.controller<span class="Apple-tab-span" style="white-space:pre"> </span>0</div>
<div>cert<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>compute.computeDomU05<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>network_fanout_b511cfb6546544aa83f7920169604fbe<span class="Apple-tab-span" style="white-space:pre"> </span>0</div>
<div>network.computeDomU08<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>consoleauth.controller<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>scheduler<span class="Apple-tab-span" style="white-space:pre"> </span>0</div>
<div><a href="http://glance_notifications.info">glance_notifications.info</a><span class="Apple-tab-span" style="white-space:pre"> </span>132</div><div>cinder-volume.controller<span class="Apple-tab-span" style="white-space:pre"> </span>0</div>
<div>glance_notifications.error<span class="Apple-tab-span" style="white-space:pre"> </span>1</div><div>network_fanout_a4dc31150bfd4fb3bf393f24a343ff76<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>
cert.computeDomU08<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>cinder-scheduler_fanout_b578bf0f687c4564920f3500ae2c9b8c<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>cinder-scheduler.controller<span class="Apple-tab-span" style="white-space:pre"> </span>0</div>
<div>network<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>compute_fanout_c1f1cd2ab9cc4c5b87fbfd72dd25a432<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>scheduler_fanout_8aa445468b5c4aa6b2a1a4412dbc09fc<span class="Apple-tab-span" style="white-space:pre"> </span>0</div>
<div>network_fanout_693737b6acaf4ba59c3c6cd31a70951b<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>consoleauth<span class="Apple-tab-span" style="white-space:pre"> </span>0</div><div>...done.</div>
</div><div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word">If the nova-networks are hanging they are probably stuck waiting on a lock files. I believe the lock files are in /var/lock/nova by default. You could try deleting them and restarting nova-network.</div>
</blockquote><div><br></div><div>However, there is no nova directory under /var/locks</div><div><br></div><div>Thank you,</div><div>Afef </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word"><div><br></div><div>Vish</div><div><div class="h5"><div><br><div><div>On Dec 10, 2012, at 1:32 PM, Afef Mdhaffar <<a href="mailto:mdhaffar@Mathematik.Uni-Marburg.de" target="_blank">mdhaffar@Mathematik.Uni-Marburg.de</a>> wrote:</div>
<br><blockquote type="cite"><br><br><div class="gmail_quote">2012/12/10 Vishvananda Ishaya <span dir="ltr"><<a href="mailto:vishvananda@gmail.com" target="_blank">vishvananda@gmail.com</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I don't see any errors in your network log. are nova-network and nova-compute running on the same host with the same config file? </blockquote><div><br></div><div>Yes, nova-network and nova-compute are running on the same node -- this is the compute node</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">It looks like it isn't recieving a message. Are you running another nova-network that is picking up the message on another host?<br>
</blockquote><div><br></div><div>I have another nova-network running on the cloud controller.</div><div><br></div><div>The platform was composed of 9 nodes (1 cloud controller + 8 compute nodes) and was running till friday (when one of my colleagues has installed some updates on our gateway). each node (including the controller) runs nova-network and nova-compute.</div>
<div>Then (after updating the gateway), only the cloud controller and one compute node are still correctly running. and the rest of the nodes returns the same error - related in my opinion to nova-network. </div><div><br>
</div><div>Any ideas how to fix that? </div><div><br></div><div>Thank you,</div><div>Afef</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Vish<br>
<div><div><br>
On Dec 10, 2012, at 12:18 PM, Afef MDHAFFAR <<a href="mailto:afef.mdhaffar@gmail.com" target="_blank">afef.mdhaffar@gmail.com</a>> wrote:<br>
<br>
> Hi all,<br>
><br>
> I am facing an boring issue since Friday: nova-network is unable to start dnsmasq. Therefore the xapi0 interface is not created.<br>
> Then, it is impossible to launch VMs. nova-network does not answer requests and nova-compute returns an RPC timeout.<br>
> Please help me to solve this problem<br>
> I am using openstack folsom release + XCP + ubuntu 12.04<br>
><br>
> log file of nova-compute and nova-network are attached to this e-mail.<br>
><br>
> Thank you,<br>
> Afef<br>
><br>
><br>
><br>
><br>
</div></div>> <nova-compute.txt><nova-network.log>_______________________________________________<br>
> Mailing list: <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
> Post to : <a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>
> Unsubscribe : <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
> More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
<br>
</blockquote></div><br>
</blockquote></div><br></div></div></div></div><br>_______________________________________________<br>
Mailing list: <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
Post to : <a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>
Unsubscribe : <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br>Afef MDHAFFAR<br><a href="http://www.redcad.org/members/mdhaffar/">http://www.redcad.org/members/mdhaffar/</a><br><br><br><br>