[Openstack] cinder - httplib.BadStatusLine

Sergey Motovilovets motovilovets.sergey at gmail.com
Tue May 20 18:33:42 UTC 2014

Hello everyone.

I'm testing Icehouse on Ubuntu 14.04 nodes, and seems like it works ok,
except for one thing - cinder randomly gives me unpleasant surprises which
from a perspective of nova appears as:

Max retries exceeded with url:
(Caused by <class 'httplib.BadStatusLine'>: '')

at approximately the same time I see following stuff in cinder-volume.log:

2014-05-20 13:08:15.139 16400 ERROR oslo.messaging._drivers.impl_rabbit [-]
Failed to consume message from queue: Socket closed
stacktrace goes here
2014-05-20 13:08:15.139 16400 TRACE oslo.messaging._drivers.impl_rabbit
IOError: Socket closed
2014-05-20 13:08:15.139 16400 TRACE oslo.messaging._drivers.impl_rabbit
2014-05-20 13:08:15.141 16400 INFO oslo.messaging._drivers.impl_rabbit [-]
Reconnecting to AMQP server on
2014-05-20 13:08:15.142 16400 INFO oslo.messaging._drivers.impl_rabbit [-]
Delaying reconnect for 1.0 seconds...
2014-05-20 13:08:16.184 16400 INFO oslo.messaging._drivers.impl_rabbit [-]
Connected to AMQP server on

That's the only suspicious thing I see in logs. is HAProxy balancing between 2 controllers with rabbitmq.
HAProxy config for Rabbit:

listen rabbitmq-cluster
        mode    tcp
option  tcpka
 balance roundrobin
        server controller01 check inter 5000 rise 2 fall 3
        server controller02 check inter 5000 rise 2 fall 3

I suppose that the reason might be HAProxy dropping rabbitmq connections
due to timeouts or something, but I have client and server timeouts set
much higher then the interval when separate stacktraces appear in log.

Have anyone faced these troubles? Any suggestions regarding optimization of
haproxy conf or something?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20140520/f8346cc4/attachment.html>

More information about the Openstack mailing list