[Openstack-operators] 100% CPU and hangs if syslog is restarted

Tim Bell Tim.Bell at cern.ch
Thu May 28 18:10:25 UTC 2015


> -----Original Message-----
> From: Christian Schwede [mailto:cschwede at redhat.com]
> Sent: 28 May 2015 20:03
> To: George Shuklin; openstack-operators at lists.openstack.org
> Subject: Re: [Openstack-operators] 100% CPU and hangs if syslog is restarted
> 
> On 28.05.15 18:56, George Shuklin wrote:
> > Today we've discover a very serious bug in juno:
> > https://bugs.launchpad.net/nova/+bug/1459726
> >
> > In short: if you're using syslog, and restart rsyslog, all APIs
> > processes will eventually stuck with 100% CPU usage without doing
> > anything.
> >
> > Is anyone hits this bug before? It looks like very nasty.
> 
> Yes, the Swift project hit this bug in the past too, though it was fixed awhile ago:
> 

Using UDP is a great workaround but it does not feel like a fix... can't the daemons realise that the syslog socket is not alive and reconnect. Given it affects most of the OpenStack projects, a fix inside one of the oslo logging libraries (if the error can be detected there) would be great.

> https://bugs.launchpad.net/swift/+bug/780025
> https://review.openstack.org/#/c/8841/
> 
> Summarized: using UDP instead of a unix socket fixes this.
> 
> -- Christian
> 
> 
> 
> 
> 

Tim

> 
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators



More information about the OpenStack-operators mailing list