Open Stack

Hi Joe,

Yes, same problem here running Ubuntu 14.04.

The symptom is nova-api, nova-conductor and glance-api eating all CPU
without responding to API requests anymore.

It is possible to reproduce it thanks to the following script.

https://gist.github.com/dbishop/7a2e224f3aafea1a1fc3

François

On 11. 10. 14 00:40, Joe Topjian wrote:
> Hello,
> 
> This morning we noticed various nova, glance, and keystone services (not
> cinder or swift) not working in two different clouds and required a restart.
> 
> We thought it was a network issue since one of the only commonalities
> between the two clouds was that they are on the same network.
> 
> Then later in the day I logged into a test cloud on a totally separate
> network and had the same problem. 
> 
> Looking at all three environments, the commonality is now that they have
> Ubuntu security updates automatically applied in the morning and this
> morning rsyslog was patched and restarted.
> 
> I found this oslo bug that kind of sounds like the issue we saw:
> 
> https://bugs.launchpad.net/oslo.log/+bug/1076466
> 
> Doing further investigation, log files do indeed show a lack of entries
> for various services/daemons until they were restarted.
> 
> Has anyone else run into this? Maybe even this morning, too? :)
> 
> Thanks,
> Joe
> 
> 
> 
> 
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>