[Openstack-operators] Slow DNS resolver causing problems for OSLO

Abel Lopez alopgeek at gmail.com
Tue Aug 5 00:21:19 UTC 2014


Seen this before outside of openstack, next time you detect DNS server
problems, it may behoove you to take it out of resolv.conf and restart
NSCD. Also, having NSCD enabled may help in the future as those records
would most likely be cached.

On Monday, August 4, 2014, George Shuklin <george.shuklin at gmail.com> wrote:

> Hello.
>
> In our installation all endpoints is defined by DNS names. Zone is hosted
> by two DNS servers, and both of them are stated in /etc/resolv.conf on each
> node.
>
> Today on of the two DNS servers fails badly due hardware malfunction and
> stops to reply to any network traffic.
>
> Theoretically, it should cause only small delays in operations - second
> DNS server is alive and reply normally.
>
> But in practice I found that every component of openstack starts to
> cripple up to level of 500 errors from api's (nova, neutron, etc). I didn't
> finish all logs reading, but it seems that slow DNS resolving causing
> congestion in connection pools of components, probably to keystone for
> token validation.
>
> It was not just 'slow', it was pure '500 errors' in API, problems with
> nova/neutron/ceilometer interoperations and so on.
>
> I'll continue to read logs tomorrow, but it really strange.
>
> Any ideas/comments?
>
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20140804/7d3d9bb0/attachment.html>


More information about the OpenStack-operators mailing list