[openstack-dev] [nova] nova-compute not re-establishing connectivity after controller switchover

John Dewey john at dewey.ws
Tue Mar 25 03:09:46 UTC 2014


Jay had responded to a similar issue [1] some time ago (I swear I saw talk of this last week but can’t find the newer thread).  Since the posting referenced we also found rabbit 3.2.x with esl erlang helped a ton.

tl;dr It is a client issue.  See the thread for further details.


[1] http://lists.openstack.org/pipermail/openstack/2013-August/000934.html  


On Monday, March 24, 2014 at 10:40 AM, Chris Friesen wrote:

> On 03/24/2014 11:31 AM, Chris Friesen wrote:
>  
> > It looks like we're raising
> >  
> > RecoverableConnectionError: connection already closed
> >  
> > down in /usr/lib64/python2.7/site-packages/amqp/abstract_channel.py, but
> > nothing handles it.
> >  
> > It looks like the most likely place that should be handling it is
> > nova.openstack.common.rpc.impl_kombu.Connection.ensure().
> >  
> >  
> > In the current oslo.messaging code the ensure() routine explicitly
> > handles connection errors (which RecoverableConnectionError is) and
> > socket timeouts--the ensure() routine in Havana doesn't do this.
> >  
>  
>  
> I misread the code, ensure() in Havana does in fact monitor socket  
> timeouts, but it doesn't handle connection errors.
>  
> It looks like support for handling connection errors was added to  
> oslo.messaging just recently in git commit 0400cbf. The git commit  
> comment talks about clustered rabbit nodes and mirrored queues which  
> doesn't apply to our scenario, but I suspect it would probably fix the  
> problem that we're seeing as well.
>  
> Chris
>  
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org (mailto:OpenStack-dev at lists.openstack.org)
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>  
>  


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140324/27184630/attachment.html>


More information about the OpenStack-dev mailing list