theheadofabroom theheadofabroom - 3 months ago 13
Python Question

getting sphinx to recognise correct signature

I've been trying to get my documentation in order for an open source project I'm working on, which involves a mirrored client and server API. To this end I have created a decorator that can most of the time be used to document a method that simply performs validation on its input. You can find a class full of these methods here and the decorator's implementation here.

The decorator, as you can see uses

functools.wraps
to preserve the docstring, and I thought also the signature, however the source code vs the generated documentation looks like this:

Source:source code

vs

Docs: sphinx docs

Does anyone know any way to have
setH
's generated documentation show the correct call signature? (without having a new decorator for each signature - there are hudreds of methods I need to mirror)

I've found a workaround which involved having the decorator not changing the unbound method, but having the class mutate the method at binding time (object instantiation) - this seems like a hack though, so any comments on this, or alternative ways of doing this, would be appreciated.

Answer

I'd like to avoid reliance on too muck outside of the standard library, so while I have looked at the Decorator module, I have mainly tried to reproduce its functionality.... Unsuccessfully...

So I took a look at the problem from another angle, and now I have a partially working solution, which can mainly be described by just looking at this commit. It's not perfect as it relies on using partial, which clobbers the help in the REPL. The idea is that instead of replacing the function to which the decorator is applied, it is augmented with attributes.

+def s_repr(obj):
+    """ :param obj: object """
+    return (repr(obj) if not isinstance(obj, SikuliClass)
+            else "self._get_jython_object(%r)" % obj._str_get)
+
+
 def run_on_remote(func):
     ...
-    func.s_repr = lambda obj: (repr(obj)
-                               if not isinstance(obj, SikuliClass) else
-                               "self._get_jython_object(%r)" % obj._str_get)
-
-    def _inner(self, *args):
-        return self.remote._eval("self._get_jython_object(%r).%s(%s)" % (
-            self._id,
-            func.__name__,
-            ', '.join([func.s_repr(x) for x in args])))
-
-    func.func = _inner
+    gjo = "self._get_jython_object"
+    func._augment = {
+        'inner': lambda self, *args: (self.remote._eval("%s(%r).%s(%s)"
+                                      % (gjo, self._id, func.__name__,
+                                         ', '.join([s_repr(x)for x in args]))))
+    }

     @wraps(func)
     def _outer(self, *args, **kwargs):
         func(self, *args, **kwargs)
-        if hasattr(func, "arg"):
-            args, kwargs = func.arg(*args, **kwargs), {}
-        result = func.func(*args, **kwargs)
-        if hasattr(func, "post"):
+        if "arg" in func._augment:
+            args, kwargs = func._augment["arg"](self, *args, **kwargs), {}
+        result = func._augment['inner'](self, *args, **kwargs)
+        if "post" in func._augment:
             return func.post(result)
         else:
             return result

     def _arg(arg_func):
-        func.arg = arg_func
-        return _outer
+        func._augment['arg'] = arg_func
+        return func

     def _post(post_func):
-        func.post = post_func
-        return _outer
+        func._augment['post'] = post_func
+        return func

     def _func(func_func):
-        func.func = func_func
-        return _outer
-    _outer.arg = _arg
-    _outer.post = _post
-    _outer.func = _func
-    return _outer
+        func._augment['inner'] = func_func
+        return func
+
+    func.arg  = _outer.arg = _arg
+    func.post = _outer.post = _post
+    func.func = _outer.func = _func
+    func.run  = _outer.run = _outer
+    return func

So this doesn't actually change the unbound method, ergo the generated documentation stays the same. The second part of the trickery occurs at class initialisation.

 class ClientSikuliClass(ServerSikuliClass):
     """ Base class for types based on the Sikuli native types """
     ...
     def __init__(self, remote, server_id, *args, **kwargs):
         """
         :type server_id: int
         :type remote: SikuliClient
         """
         super(ClientSikuliClass, self).__init__(None)
+        for key in dir(self):
+            try:
+                func = getattr(self, key)
+            except AttributeError:
+                pass
+            else:
+                try:
+                    from functools import partial, wraps
+                    run = wraps(func.run)(partial(func.run, self))
+                    setattr(self, key, run)
+                except AttributeError:
+                    pass
         self.remote = remote
         self.server_id = server_id

So at the point where an instance of any class inheriting ClientSikuliClass is instantiated, an attempt is made to take the run property of each attribute of that instance and make that what is returned on attempting to get that attribute, and so the bound method is now a partially applied _outer function.

So the issues with this are multiple:

  1. Using partial at initilaisation results in losing the bound method information.
  2. I worry about clobbering attributes that just so happen to have a run attribute...

So while I have an answer to my own question, I'm not quite satisfied by it.


Update

Ok so after a bit more work I ended up with this:

 class ClientSikuliClass(ServerSikuliClass):
     """ Base class for types based on the Sikuli native types """
     ...
     def __init__(self, remote, server_id, *args, **kwargs):
         """
         :type server_id: int
         :type remote: SikuliClient
         """
         super(ClientSikuliClass, self).__init__(None)
-        for key in dir(self):
+
+        def _apply_key(key):
             try:
                 func = getattr(self, key)
+                aug = func._augment
+                runner = func.run
             except AttributeError:
-                pass
-            else:
-                try:
-                    from functools import partial, wraps
-                    run = wraps(func.run)(partial(func.run, self))
-                    setattr(self, key, run)
-                except AttributeError:
-                    pass
+                return
+
+            @wraps(func)
+            def _outer(*args, **kwargs):
+                return runner(self, *args, **kwargs)
+
+            setattr(self, key, _outer)
+
+        for key in dir(self):
+            _apply_key(key)
+
         self.remote = remote
         self.server_id = server_id

This prevents the loss of the documentation on the object. You'll also see that the func._augment attribute is accessed, even though it is not used, so that if it does not exist the object attribute will not be touched.

I'd be interested if anyone had any comments on this?