<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Jul 12, 2013 at 2:40 PM, Brant Knudson <span dir="ltr"><<a href="mailto:blk@acm.org" target="_blank">blk@acm.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><br></div>Somewhat offtopic, but others might find it interesting. On another project I used a different technique for exceptions, where the original exception is "enhanced" with new fields as it propagates up the stack to where it's handled. In this way all the information needed to handle the exception is available (even if it's just to log it).<br>
<br>Often there's some information that would be useful for the exception handler that isn't available at the place where the exception is raised. The classic example is an error writing to a file where you'd like to know the filename but all you've got is the file descriptor. An example from an OpenStack REST API might be that you get an exception and you'd like to log a message that includes the resource path, but the resource path isn't known at this point.<br>
<div><br></div><div>So rather than logging the exception when it happens, enhance the exception, re-raise it, and only once it's got all the information where you can print out a useful log message you log it.<br></div>
<div><br></div>Here's a short example to illustrate. An exception is raised by f1(), but at this point we don't know the status code that the server should respond with or the request path which would be useful in a log message. These bits of information are added as the exception propagates and then where it's caught we've got all the information for a useful log message.<br>
<div><br>def f1():<br> raise Exception('something')<br><br><br>def f2():<br> try:<br> f1()<br> except Exception as e:<br> e.status_code = 403<br> raise<br><br><br>def do_tokens():<br> try:<br> f2()<br>
except Exception as e:<br> e.req_url = '/v3/tokens'<br> raise<br><br><br>status_code = 200<br>try:<br> do_tokens()<br>except Exception as e:<br> status_code = getattr(e, 'status_code', 500) <br>
req_url = getattr(e, 'req_url', 'unknown')<br><br>if status_code != 200:<br> print 'Got %s from %s' % (status_code, req_url)<br></div><div> # build an error document for the response.<br></div>
</div></blockquote><div><br></div><div><br></div><div style>One problem with this approach is it spreads knowledge of the error construction up and down the stack through different layers of the application, and that brings with it assumptions about the implementation at the different layers. For example, should the application code above know that do_tokens() is making web requests to URLs? Why does it need that information?</div>
<div style><br></div><div style>SRP-ly,</div><div style>Doug</div><div style><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>
</div>
<div><br></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Jul 12, 2013 at 12:25 PM, Nachi Ueno <span dir="ltr"><<a href="mailto:nachi@ntti3.com" target="_blank">nachi@ntti3.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi folks<br>
<br>
> Monty<br>
Thank you for your adding good topic for this.<br>
<br>
I agree, hiding true cause by exception get hard to debug<br>
<br>
For, example, Sqlalchemy gives me this error for general sql errors..<br>
"exceptions must be old-style classes or derived from BaseException, not str"<br>
<br>
(It should be hidden from REST API users though)<br>
<br>
Also, I agree with you about log policies.<br>
sometimes 404 get warn log..<br>
<br>
IMO, The log more than warn level should help Operators.<br>
also If the exception is truly, the exceptional case which needs help<br>
of operators.<br>
It should be logged as error level.<br>
<br>
Thanks<br>
Nachi<br>
<br>
<br>
<br>
2013/7/12 Dolph Mathews <<a href="mailto:dolph.mathews@gmail.com" target="_blank">dolph.mathews@gmail.com</a>>:<br>
<div><div>><br>
> On Fri, Jul 12, 2013 at 10:09 AM, Monty Taylor <<a href="mailto:mordred@inaugust.com" target="_blank">mordred@inaugust.com</a>> wrote:<br>
>><br>
>><br>
>><br>
>> On 07/11/2013 05:20 AM, Mark McLoughlin wrote:<br>
>> > On Wed, 2013-07-10 at 19:49 -0400, Monty Taylor wrote:<br>
>> >> I'd like top-post and hijack this thread for another exception related<br>
>> >> thing:<br>
>> >><br>
>> >> a) Anyone writing code such as:<br>
>> >><br>
>> >> try:<br>
>> >> blah()<br>
>> >> except SomeException:<br>
>> >> raise SomeOtherExceptionLeavingOutStackContextFromSomeException<br>
>> >><br>
>> >> should be mocked ruthlessly.<br>
>> ><br>
>> > Ok, mock me ruthlessly then.<br>
>><br>
>> Mock. Mock. Mock. Mock.<br>
>><br>
>> > Part of designing any API is specifying what predictable exceptions it<br>
>> > will raise. For any predictable error condition, you don't want callers<br>
>> > to have to catch random exceptions from the underlying libraries you<br>
>> > might be calling into.<br>
>><br>
>> Absolutely. But you don't want to go so overboard that you remove the<br>
>> ability for a developer to debug it. More importantly, INSIDE of server<br>
>> code, we're not designing fun apis for the consumption of people we<br>
>> don't know. There is clearly an API, but we are the consumers of our own<br>
>> API.<br>
><br>
><br>
> Lies! Write everything to be intuitive for new contributors or we won't have<br>
> any :(<br>
><br>
>><br>
>><br>
>> > Say if I was designing an image downloading API, I'd do something like<br>
>> > this:<br>
>> ><br>
>> > <a href="https://gist.github.com/markmc/5973868" target="_blank">https://gist.github.com/markmc/5973868</a><br>
>> ><br>
>> > Assume there's a tonne more stuff that the API would do. You don't want<br>
>> > callers to have to catch socket.error exceptions and whatever other<br>
>> > exceptions might be thrown.<br>
>> ><br>
>> > That works out as:<br>
>> ><br>
>> > Traceback (most recent call last):<br>
>> > File "t.py", line 20, in <module><br>
>> > download_image('localhost', 3366, 'foobar')<br>
>> > File "t.py", line 18, in download_image<br>
>> > raise ImageDownloadFailure(host, port, path, e.strerror)<br>
>> > __main__.ImageDownloadFailure: Failed to download foobar from<br>
>> > localhost:3366: Connection refused<br>
>> ><br>
>> > Which is a pretty clear exception.<br>
>><br>
>> Right, but what if the message from the exception does not give you<br>
>> enough information to walk down the stack and see it...<br>
>><br>
>> Also, I have more than once seen:<br>
>><br>
>> class MyIOError(Exception):<br>
>> pass<br>
>><br>
>> try:<br>
>> s = socket.create_connection((host, port))<br>
>> except socket.error:<br>
>> raise MyIOError("something went wrong!")<br>
>><br>
>> Which is an extreme example of where my rage comes from, but not a made<br>
>> up example. I have, actually, seen code exactly like that - in Nova.<br>
>><br>
>> BTW - I'd have download_image return None if it can't download the<br>
>> image, and I'd have it either deal with the socket.error or not locally<br>
>> at the source. But now we're nitpicking.<br>
>><br>
>> > But I think what you're saying is missing is the stack trace from the<br>
>> > underlying exception.<br>
>><br>
>> YES - and I'll let David's response respond to the details of the rest...<br>
>><br>
>> But essentially, what I was really mocking earlier is throwing a new<br>
>> exception in such a way that you lose the exception propagation path. So<br>
>> if you throw an exception inside of an except block, you should be sure<br>
>> that you're passing on all of the info, because if a developer gets an<br>
>> exception, it's quite likely that they want to know how to fix it. :)<br>
>><br>
>> > As I understood it, Python doesn't have a way of chaining exceptions<br>
>> > like this but e.g. Java does. A little bit more poking right now shows<br>
>> > up this:<br>
>> ><br>
>> > <a href="http://www.python.org/dev/peps/pep-3134/" target="_blank">http://www.python.org/dev/peps/pep-3134/</a><br>
>> ><br>
>> > i.e. we can't do the right thing until Python 3, where we'd do:<br>
>> ><br>
>> > def download_image(host, port, path):<br>
>> > try:<br>
>> > s = socket.create_connection((host, port))<br>
>> > except socket.error as e:<br>
>> > raise ImageDownloadFailure(host, port, path, e.strerror) from e<br>
>><br>
>> This will certainly be cleaner to write and read.<br>
>><br>
>> > I haven't read the PEP in detail yet, though.<br>
>> ><br>
>> > Cheers,<br>
>> > Mark.<br>
>> ><br>
>> ><br>
>> > _______________________________________________<br>
>> > OpenStack-dev mailing list<br>
>> > <a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
>> > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>> ><br>
>><br>
>> _______________________________________________<br>
>> OpenStack-dev mailing list<br>
>> <a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
><br>
><br>
><br>
><br>
> --<br>
><br>
> -Dolph<br>
><br>
> _______________________________________________<br>
> OpenStack-dev mailing list<br>
> <a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
><br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div>