Open Stack

Could also try changing the timeout option in resolve.conf(5).

John

On Monday, August 4, 2014 at 5:21 PM, Abel Lopez wrote:

> Seen this before outside of openstack, next time you detect DNS server problems, it may behoove you to take it out of resolv.conf and restart NSCD. Also, having NSCD enabled may help in the future as those records would most likely be cached. 
> 
> On Monday, August 4, 2014, George Shuklin <george.shuklin at gmail.com (mailto:george.shuklin at gmail.com)> wrote:
> > Hello.
> > 
> > In our installation all endpoints is defined by DNS names. Zone is hosted by two DNS servers, and both of them are stated in /etc/resolv.conf on each node.
> > 
> > Today on of the two DNS servers fails badly due hardware malfunction and stops to reply to any network traffic.
> > 
> > Theoretically, it should cause only small delays in operations - second DNS server is alive and reply normally.
> > 
> > But in practice I found that every component of openstack starts to cripple up to level of 500 errors from api's (nova, neutron, etc). I didn't finish all logs reading, but it seems that slow DNS resolving causing congestion in connection pools of components, probably to keystone for token validation.
> > 
> > It was not just 'slow', it was pure '500 errors' in API, problems with nova/neutron/ceilometer interoperations and so on.
> > 
> > I'll continue to read logs tomorrow, but it really strange.
> > 
> > Any ideas/comments?
> > 
> > _______________________________________________
> > OpenStack-operators mailing list
> > OpenStack-operators at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org (mailto:OpenStack-operators at lists.openstack.org)
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
> 
> 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20140804/3d44b2d9/attachment-0001.html>