<html><body>
<p><font size="2" face="sans-serif">Yes, we're still on a code level that uses httplib2. I noticed that as well, but wasn't sure if that would really</font><br>
<font size="2" face="sans-serif">help here as it seems like an ssl thing itself. But... who knows?? I'm not sure how consistently we can</font><br>
<font size="2" face="sans-serif">recreate this, but if we can, I'll try using that patch to use requests and see if that helps.<br>
<br>
</font><br>
<br>
<tt><font size="2">"Armando M." <armamig@gmail.com> wrote on 05/29/2014 11:52:34 AM:<br>
<br>
> From: "Armando M." <armamig@gmail.com></font></tt><br>
<tt><font size="2">> To: "OpenStack Development Mailing List (not for usage questions)" <br>
> <openstack-dev@lists.openstack.org>, </font></tt><br>
<tt><font size="2">> Date: 05/29/2014 11:58 AM</font></tt><br>
<tt><font size="2">> Subject: Re: [openstack-dev] [neutron] Supporting retries in neutronclient</font></tt><br>
<tt><font size="2">> <br>
> Hi Paul,<br>
> <br>
> Just out of curiosity, I am assuming you are using the client that<br>
> still relies on httplib2. Patch [1] replaced httplib2 with requests,<br>
> but I believe that a new client that incorporates this change has not<br>
> yet been published. I wonder if the failures you are referring to<br>
> manifest themselves with the former http library rather than the<br>
> latter. Could you clarify?<br>
> <br>
> Thanks,<br>
> Armando<br>
> <br>
> [1] - <a href="https://review.openstack.org/#/c/89879/">https://review.openstack.org/#/c/89879/</a><br>
> <br>
> On 29 May 2014 17:25, Paul Ward <wpward@us.ibm.com> wrote:<br>
> > Well, for my specific error, it was an intermittent ssl handshake error<br>
> > before the request was ever sent to the<br>
> > neutron-server. In our case, we saw that 4 out of 5 resize operations<br>
> > worked, the fifth failed with this ssl<br>
> > handshake error in neutronclient.<br>
> ><br>
> > I certainly think a GET is safe to retry, and I agree with your statement<br>
> > that PUTs and DELETEs probably<br>
> > are as well. This still leaves a change in nova needing to be made to<br>
> > actually a) specify a conf option and<br>
> > b) pass it to neutronclient where appropriate.<br>
> ><br>
> ><br>
> > Aaron Rosen <aaronorosen@gmail.com> wrote on 05/28/2014 07:38:56 PM:<br>
> ><br>
> >> From: Aaron Rosen <aaronorosen@gmail.com><br>
> ><br>
> ><br>
> >> To: "OpenStack Development Mailing List (not for usage questions)"<br>
> >> <openstack-dev@lists.openstack.org>,<br>
> >> Date: 05/28/2014 07:44 PM<br>
> ><br>
> >> Subject: Re: [openstack-dev] [neutron] Supporting retries in neutronclient<br>
> >><br>
> >> Hi,<br>
> >><br>
> >> I'm curious if other openstack clients implement this type of retry<br>
> >> thing. I think retrying on GET/DELETES/PUT's should probably be okay.<br>
> >><br>
> >> What types of errors do you see in the neutron-server when it fails<br>
> >> to respond? I think it would be better to move the retry logic into<br>
> >> the server around the failures rather than the client (or better yet<br>
> >> if we fixed the server :)). Most of the times I've seen this type of<br>
> >> failure is due to deadlock errors caused between (sqlalchemy and<br>
> >> eventlet *i think*) which cause the client to eventually timeout.<br>
> >><br>
> >> Best,<br>
> >><br>
> >> Aaron<br>
> >><br>
> ><br>
> >> On Wed, May 28, 2014 at 11:51 AM, Paul Ward <wpward@us.ibm.com> wrote:<br>
> >> Would it be feasible to make the retry logic only apply to read-only<br>
> >> operations? This would still require a nova change to specify the<br>
> >> number of retries, but it'd also prevent invokers from shooting<br>
> >> themselves in the foot if they call for a write operation.<br>
> >><br>
> >><br>
> >><br>
> >> Aaron Rosen <aaronorosen@gmail.com> wrote on 05/27/2014 09:40:00 PM:<br>
> >><br>
> >> > From: Aaron Rosen <aaronorosen@gmail.com><br>
> >><br>
> >> > To: "OpenStack Development Mailing List (not for usage questions)"<br>
> >> > <openstack-dev@lists.openstack.org>,<br>
> >> > Date: 05/27/2014 09:44 PM<br>
> >><br>
> >> > Subject: Re: [openstack-dev] [neutron] Supporting retries in<br>
> >> > neutronclient<br>
> >> ><br>
> >> > Hi,<br>
> >><br>
> >> ><br>
> >> > Is it possible to detect when the ssl handshaking error occurs on<br>
> >> > the client side (and only retry for that)? If so I think we should<br>
> >> > do that rather than retrying multiple times. The danger here is<br>
> >> > mostly for POST operations (as Eugene pointed out) where it's<br>
> >> > possible for the response to not make it back to the client and for<br>
> >> > the operation to actually succeed.<br>
> >> ><br>
> >> > Having this retry logic nested in the client also prevents things<br>
> >> > like nova from handling these types of failures individually since<br>
> >> > this retry logic is happening inside of the client. I think it would<br>
> >> > be better not to have this internal mechanism in the client and<br>
> >> > instead make the user of the client implement retry so they are<br>
> >> > aware of failures.<br>
> >> ><br>
> >> > Aaron<br>
> >> ><br>
> >><br>
> >> > On Tue, May 27, 2014 at 10:48 AM, Paul Ward <wpward@us.ibm.com> wrote:<br>
> >> > Currently, neutronclient is hardcoded to only try a request once in<br>
> >> > retry_request by virtue of the fact that it uses self.retries as the<br>
> >> > retry count, and that's initialized to 0 and never changed. We've<br>
> >> > seen an issue where we get an ssl handshaking error intermittently<br>
> >> > (seems like more of an ssl bug) and a retry would probably have<br>
> >> > worked. Yet, since neutronclient only tries once and gives up, it<br>
> >> > fails the entire operation. Here is the code in question:<br>
> >> ><br>
> >> > <a href="https://github.com/openstack/python-neutronclient/blob/master/">https://github.com/openstack/python-neutronclient/blob/master/</a><br>
> >> > neutronclient/v2_0/client.py#L1296<br>
> >> ><br>
> >> > Does anybody know if there's some explicit reason we don't currently<br>
> >> > allow configuring the number of retries? If not, I'm inclined to<br>
> >> > propose a change for just that.<br>
> >> ><br>
> >> > _______________________________________________<br>
> >> > OpenStack-dev mailing list<br>
> >> > OpenStack-dev@lists.openstack.org<br>
> >> > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
> >><br>
> >> > _______________________________________________<br>
> >> > OpenStack-dev mailing list<br>
> >> > OpenStack-dev@lists.openstack.org<br>
> >> > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
> >><br>
> >> _______________________________________________<br>
> >> OpenStack-dev mailing list<br>
> >> OpenStack-dev@lists.openstack.org<br>
> >> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
> ><br>
> >> _______________________________________________<br>
> >> OpenStack-dev mailing list<br>
> >> OpenStack-dev@lists.openstack.org<br>
> >> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
> ><br>
> ><br>
> > _______________________________________________<br>
> > OpenStack-dev mailing list<br>
> > OpenStack-dev@lists.openstack.org<br>
> > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
> ><br>
> <br>
> _______________________________________________<br>
> OpenStack-dev mailing list<br>
> OpenStack-dev@lists.openstack.org<br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
> <br>
</font></tt></body></html>