Open Stack

Well, for my specific error, it was an intermittent ssl handshake error
before the request was ever sent to the
neutron-server.  In our case, we saw that 4 out of 5 resize operations
worked, the fifth failed with this ssl
handshake error in neutronclient.

I certainly think a GET is safe to retry, and I agree with your statement
that PUTs and DELETEs probably
are as well.  This still leaves a change in nova needing to be made to
actually a) specify a conf option and
b) pass it to neutronclient where appropriate.


Aaron Rosen <aaronorosen at gmail.com> wrote on 05/28/2014 07:38:56 PM:

> From: Aaron Rosen <aaronorosen at gmail.com>
> To: "OpenStack Development Mailing List (not for usage questions)"
> <openstack-dev at lists.openstack.org>,
> Date: 05/28/2014 07:44 PM
> Subject: Re: [openstack-dev] [neutron] Supporting retries in
neutronclient
>
> Hi,
>
> I'm curious if other openstack clients implement this type of retry
> thing. I think retrying on GET/DELETES/PUT's should probably be okay.
>
> What types of errors do you see in the neutron-server when it fails
> to respond? I think it would be better to move the retry logic into
> the server around the failures rather than the client (or better yet
> if we fixed the server :)). Most of the times I've seen this type of
> failure is due to deadlock errors caused between (sqlalchemy and
> eventlet *i think*) which cause the client to eventually timeout.
>
> Best,
>
> Aaron
>

> On Wed, May 28, 2014 at 11:51 AM, Paul Ward <wpward at us.ibm.com> wrote:
> Would it be feasible to make the retry logic only apply to read-only
> operations?  This would still require a nova change to specify the
> number of retries, but it'd also prevent invokers from shooting
> themselves in the foot if they call for a write operation.
>
>
>
> Aaron Rosen <aaronorosen at gmail.com> wrote on 05/27/2014 09:40:00 PM:
>
> > From: Aaron Rosen <aaronorosen at gmail.com>
>
> > To: "OpenStack Development Mailing List (not for usage questions)"
> > <openstack-dev at lists.openstack.org>,
> > Date: 05/27/2014 09:44 PM
>
> > Subject: Re: [openstack-dev] [neutron] Supporting retries in
neutronclient
> >
> > Hi,
>
> >
> > Is it possible to detect when the ssl handshaking error occurs on
> > the client side (and only retry for that)? If so I think we should
> > do that rather than retrying multiple times. The danger here is
> > mostly for POST operations (as Eugene pointed out) where it's
> > possible for the response to not make it back to the client and for
> > the operation to actually succeed.
> >
> > Having this retry logic nested in the client also prevents things
> > like nova from handling these types of failures individually since
> > this retry logic is happening inside of the client. I think it would
> > be better not to have this internal mechanism in the client and
> > instead make the user of the client implement retry so they are
> > aware of failures.
> >
> > Aaron
> >
>
> > On Tue, May 27, 2014 at 10:48 AM, Paul Ward <wpward at us.ibm.com> wrote:
> > Currently, neutronclient is hardcoded to only try a request once in
> > retry_request by virtue of the fact that it uses self.retries as the
> > retry count, and that's initialized to 0 and never changed.  We've
> > seen an issue where we get an ssl handshaking error intermittently
> > (seems like more of an ssl bug) and a retry would probably have
> > worked.  Yet, since neutronclient only tries once and gives up, it
> > fails the entire operation.  Here is the code in question:
> >
> > https://github.com/openstack/python-neutronclient/blob/master/
> > neutronclient/v2_0/client.py#L1296
> >
> > Does anybody know if there's some explicit reason we don't currently
> > allow configuring the number of retries?  If not, I'm inclined to
> > propose a change for just that.
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140529/ee9592f9/attachment.html>