Alan Alan - 24 days ago 10
Python Question

Stop at exception in my, not library code

I'm developing an app using a Python library

urllib
and it is sometimes rising exceptions due to not being able to access an URL.

However, the exception is raised almost 6 levels into the standard library stack:

/home/user/Workspace/application/main.py in call(path)
11 headers={'content-type': 'application/json'},
12 data=b'')
---> 13 resp = urllib.request.urlopen(req) ####### THIS IS MY CODE
14 return json.loads(resp.read().decode('utf-8'))

/usr/lib/python3.4/urllib/request.py in urlopen(url, data, timeout, cafile, capath, cadefault, context)
159 else:
160 opener = _opener
--> 161 return opener.open(url, data, timeout)
162
163 def install_opener(opener):

/usr/lib/python3.4/urllib/request.py in open(self, fullurl, data, timeout)
461 req = meth(req)
462
--> 463 response = self._open(req, data)
464
465 # post-process response

/usr/lib/python3.4/urllib/request.py in _open(self, req, data)
479 protocol = req.type
480 result = self._call_chain(self.handle_open, protocol, protocol +
--> 481 '_open', req)
482 if result:
483 return result

/usr/lib/python3.4/urllib/request.py in _call_chain(self, chain, kind, meth_name, *args)
439 for handler in handlers:
440 func = getattr(handler, meth_name)
--> 441 result = func(*args)
442 if result is not None:
443 return result

/usr/lib/python3.4/urllib/request.py in http_open(self, req)
1208
1209 def http_open(self, req):
-> 1210 return self.do_open(http.client.HTTPConnection, req)
1211
1212 http_request = AbstractHTTPHandler.do_request_

/usr/lib/python3.4/urllib/request.py in do_open(self, http_class, req, **http_conn_args)
1182 h.request(req.get_method(), req.selector, req.data, headers)
1183 except OSError as err: # timeout error
-> 1184 raise URLError(err)
1185 r = h.getresponse()
1186 except:

URLError: <urlopen error [Errno 111] Connection refused>


I usually run the code in
ipython3
with the
%pdb
magic turned on so in case there is an exception I can inspect it immediately. However for this I have to go down the stack 6 levels to get to my code.

Is it achievable that my app crashes pointing to my code directly?

Answer

I would go with modifying the code:

try:
    resp = urllib.request.urlopen(req)

except Exception as e:
    raise RuntimeError(e)

That way:

  • %pdb moves you to your code,
  • the original exception is preserved as argument of the "secondary" exception.

You may also monkeypatch urllib.request.urlopen() function:

class MonkeyPatchUrllib(object):
    def __enter__(self):
        self.__urlopen = urllib.request.urlopen
        urllib.request.urlopen = self
    def __exit__(self, exception_type, exception_value, traceback):
        urllib.request.urlopen = self.__urlopen
    def __call__(self, *args, **kwargs):
        try:                                  
            return self.__urlopen(*args, **kwargs)
        except Exception as e:
            raise RuntimeError(e)

Any time you have an exception raised in urlibopen() call within the context manager scope:

with MonkeyPatchUrllib():
    #your code here

%pdb will move you only 1 level away from your code.

[EDIT]

With sys.exc_info() it is possible to preserve a more verbose context of the original exception (like its traceback).