<p dir="ltr">False alarm, after more tests the issue persisted, so I switched to backup mode in the other haproxy nodes and now everything works as expected.</p>
<p dir="ltr">Thanks</p>
<div class="gmail_quote">Em 15/01/2015 12:13, "Pedro Sousa" <<a href="mailto:pgsousa@gmail.com">pgsousa@gmail.com</a>> escreveu:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Hi all,</div><div><br></div><div>the culprit was haproxy, I had "option httpchk" when I disabled this stopped having timeouts rebooting the servers. </div><div><br></div><div>Thank you all.</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jan 14, 2015 at 5:29 PM, John Dewey <span dir="ltr"><<a href="mailto:john@dewey.ws" target="_blank">john@dewey.ws</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
I would verify that the VIP failover is occurring.
</div><div><br></div><div>Your master should have the IP address. If you shut down keepalived the VIP should move to one of the others. I generally set the state to MASTER on all systems, and have one with a higher priority than the others (e.g. 100 vs 150 on others).</div><div><div><div><br></div><div><span style="color:rgb(160,160,168)">On Tuesday, January 13, 2015 at 12:18 PM, Pedro Sousa wrote:</span></div>
<blockquote type="cite" style="border-left-style:solid;border-width:1px;margin-left:0px;padding-left:10px">
<span><div><div><div dir="ltr">As expected If I reboot the Keepalived MASTER node, I get timeouts again, so my understanding is that this happens when the VIP fails over to another node. Anyone has explanation for this?<div><br></div><div>Thanks</div><div><br><div>On Tue, Jan 13, 2015 at 8:08 PM, Pedro Sousa <span dir="ltr"><<a href="mailto:pgsousa@gmail.com" target="_blank">pgsousa@gmail.com</a>></span> wrote:<br><blockquote type="cite"><div><div dir="ltr"><div>Hi,</div><div><br></div>I think I found out the issue, as I have all the 3 nodes running Keepalived as MASTER, when I reboot one of the servers, one of the VIPS failsover to it, causing the timeout issues. So I left only one server as MASTER and the other 2 as BACKUP, and If I reboot the BACKUP servers everything will work fine.<div><br></div><div>As a note aside, I don't know if this is some ARP issue because I have a similar problem with Neutron L3 running in HA Mode. If I reboot the server that is running as MASTER I loose connection to my floating IPS because the switch doesn't know yet that the Mac Addr has changed. To everything start working I have to ping an outside host like google from an instance.<br></div><div><div><br></div><div>Maybe someone could share some experience on this,</div><div><br></div><div>Thank you for your help.</div><div><br><div><br></div><div><br></div></div></div></div><div><div><div><br><div>On Tue, Jan 13, 2015 at 7:18 PM, Pedro Sousa <span dir="ltr"><<a href="mailto:pgsousa@gmail.com" target="_blank">pgsousa@gmail.com</a>></span> wrote:<br><blockquote type="cite"><div><div dir="ltr"><div>Jesse,</div><div><br></div>I see a lot of these messages in glance-api:<div><br></div><div><div>2015-01-13 19:16:29.084 29269 DEBUG glance.api.middleware.version_negotiation [29d94a9a-135b-4bf2-a97b-f23b0704ee15 eb7ff2b5f0f34f51ac9ea0f75b60065d 2524b02b63994749ad1fed6f3a825c15 - - -] Unknown version. Returning version choices. process_request /usr/lib/python2.7/site-packages/glance/api/middleware/version_negotiation.py:64</div></div><div><br></div><div>While running openstack-status (glance image-list)</div><div><div><br></div><div>== Glance images ==</div><div>Error finding address for <a href="http://172.16.21.20:9292/v1/images/detail?sort_key=name&sort_dir=asc&limit=20" target="_blank">http://172.16.21.20:9292/v1/images/detail?sort_key=name&sort_dir=asc&limit=20</a>: HTTPConnectionPool(host='172.16.21.20', port=9292): Max retries exceeded with url: /v1/images/detail?sort_key=name&sort_dir=asc&limit=20 (Caused by <class 'httplib.BadStatusLine'>: '')</div></div><div><br></div><div><br></div><div>Thanks</div><div><br></div></div><div><div><div><br><div>On Tue, Jan 13, 2015 at 6:52 PM, Jesse Keating <span dir="ltr"><<a href="mailto:jlk@bluebox.net" target="_blank">jlk@bluebox.net</a>></span> wrote:<br><blockquote type="cite"><div><span>On 1/13/15 10:42 AM, Pedro Sousa wrote:<br>
</span><blockquote type="cite"><div><span>
Hi<br>
<br>
<br>
I've changed some haproxy confs, now I'm getting a different error:<br>
<br></span>
*== Nova networks ==*<br>
*ERROR (ConnectionError): HTTPConnectionPool(host='172.<u></u>16.21.20',<span><br>
port=8774): Max retries exceeded with url:<br>
/v2/<u></u>2524b02b63994749ad1fed6f3a825c<u></u>15/os-networks (Caused by <class<br></span>
'httplib.BadStatusLine'>: '')*<br>
*== Nova instance flavors ==*<span><br>
<br>
If I restart my openstack services everything will start working.<br>
<br>
I'm attaching my new haproxy conf.<br>
<br>
<br>
Thanks<br>
<br>
</span></div></blockquote><br>
Sounds like your services are losing access to something, like rabbit or the database. What do your service logs show prior to restart? Are they throwing any errors?<span><font color="#888888"><br>
<br>
<br>
-- <br>
-jlk</font></span><div><div><br>
<br>
______________________________<u></u>_________________<br>
OpenStack-operators mailing list<br>
<a href="mailto:OpenStack-operators@lists.openstack.org" target="_blank">OpenStack-operators@lists.<u></u>openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack-operators</a><br>
</div></div></div></blockquote></div><br></div>
</div></div></div></blockquote></div><br></div>
</div></div></div></blockquote></div><br></div></div>
</div></div></span>
</blockquote>
<div>
<br>
</div>
</div></div></blockquote></div><br></div>
</blockquote></div>