paul23 paul23 - 1 year ago 91
Python Question

Type hinting, union with forward references

Well in python 3 type hinting can now be used.

In my small script I wish to use type hinting. However a specific variable can be of two types. Either an

(denoting a location), or an
(from which to get the location).
The function is within a

Now the CelestialBody object isn't defined before the class so I'm using a forward reference, as described by the pep

from math import pi
import numpy as np
import math as m
from numpy import cos, sin, sqrt, power, square, arctan2, arccos, arcsin, arcsinh, radians, degrees
from scipy.optimize import *
import scipy as sp
import celestial_body as CB
import typing


def get_total_max_distance(self, ancestor_body: "CB.CelestialBody", eps=3*np.finfo(float).eps):
if self.parent == ancestor_body:
return self.apoapsis_distance
orbit_list = list(self.create_tree_branch(ancestor_body))
return orbit_list[0]._get_total_max_distance(ancestor_body.getGlobalPositionAtTime(),
orbit_list[1:], eps)

This works fine. Pycharm understands the type and it seems correct? Now I wish to change this so it understands it can take both a
and a
type. (Where the second is already declared). I tried using a union according to the pep:

def get_total_max_distance(self, ancestor_body: typing.Union["CB.CelestialBody",np.ndarray], eps=3*np.finfo(float).eps):

However this fails with the remark: "AttributeError: 'module' object has no attribute 'CelestialBody'"

full traceback:

Traceback (most recent call last):
File "C:/Users/Paul/PycharmProjects/KSP_helper/", line 5, in <module>
from celestial_body import *
File "C:\Users\Paul\PycharmProjects\KSP_helper\", line 7, in <module>
import celestial_orbit as CO
File "C:\Users\Paul\PycharmProjects\KSP_helper\", line 123, in <module>
class CelestialOrbit:
File "C:\Users\Paul\PycharmProjects\KSP_helper\", line 579, in CelestialOrbit
def get_total_max_distance(self, ancestor_body: typing.Union["CB.CelestialBody",np.ndarray], eps=3*np.finfo(float).eps):
File "C:\Python34\lib\site-packages\", line 537, in __getitem__
dict(self.__dict__), parameters, _root=True)
File "C:\Python34\lib\site-packages\", line 494, in __new__
for t2 in all_params - {t1} if not isinstance(t2, TypeVar)):
File "C:\Python34\lib\site-packages\", line 494, in <genexpr>
for t2 in all_params - {t1} if not isinstance(t2, TypeVar)):
File "C:\Python34\lib\site-packages\", line 185, in __subclasscheck__
self._eval_type(globalns, localns)
File "C:\Python34\lib\site-packages\", line 172, in _eval_type
eval(self.__forward_code__, globalns, localns),
File "<string>", line 1, in <module>
AttributeError: 'module' object has no attribute 'CelestialBody'

How would I do this?

Following Kevin's advice show a bit easier error as one expect (cause, since the celestial_orbit module is imported by the celestial_body module the CelestialBody class hasn't be instantiated when python tries to instantiate the CelestialOrbit class).

C:\Python35\python.exe C:/Users/Paul/PycharmProjects/KSP_helper/
Traceback (most recent call last):
File "C:/Users/Paul/PycharmProjects/KSP_helper/", line 5, in <module>
from celestial_body import *
File "C:\Users\Paul\PycharmProjects\KSP_helper\", line 7, in <module>
import celestial_orbit as CO
File "C:\Users\Paul\PycharmProjects\KSP_helper\", line 125, in <module>
class CelestialOrbit:
File "C:\Users\Paul\PycharmProjects\KSP_helper\", line 581, in CelestialOrbit
def get_total_max_distance(self, ancestor_body: typing.Union[CB.CelestialBody, np.ndarray], eps=3*np.finfo(float).eps):
AttributeError: module 'celestial_body' has no attribute 'CelestialBody'

The advice from Bakuriu - changing
seems to work. However this is to me highly illogical - especially since the non-union version works with the

Answer Source

Note: This bug was recently fixed and will be part of Python 3.5.3. The answer below is outdated as of that version.

The forward reference fails to resolve because your CB module reference exists, but does not have a CelestialBody attribute, so an AttributeError exception is raised. Forward reference resolving (indirectly triggered by the Union type) allows for NameError exceptions only; supposedly because that's the canonical way to determine if a name is (not yet) available.

But given that the PEP provides you with an example where forward references are used to resolve a circular dependency between two modules (and thus you'd expect AttributeErrors to occur when prematurely referencing the name), I'm a little surprised that what you tried doesn't work, actually. You almost certainly have found a bug.

What happens is that the Union[...] type checks if the elements in the union are a subclass of another type in the union, and it is that check that triggers an attempt to look up the forward reference. If 'a.A' and 'b.B' (as in the circular reference example) are to work, the forward reference check should accept AttributeError as a valid exception to handle here. In fact, any exception should be swallowed at this point, because, as the PEP states:

The string literal should contain a valid Python expression [...] and it should evaluate without errors once the module has been fully loaded.

Emphasis mine. When a Union[..] object is created, the module is not yet fully loaded, so given that any valid Python expression is permitted the code should treat any exception as indication the forward reference is not yet ready and ignore it.

The work-around would be for you to create function that turns the AttributeError into a NameError:

def _CelestialBody_forward_ref():
        return CB.CelestialBody
    except AttributeError:
        # not yet, raise NameError instead
        raise NameError('CelestialBody')

then use that in your forward references:

typing.Union['_CelestialBody_forward_ref()', np.ndarray]

This works because forward references are allowed to be any valid Python expression. Or you could make the whole Union declaration a string; it'll be evaluated later when all imports are done:

def get_total_max_distance(self, ancestor_body: "typing.Union[CB.CelestialBody,np.ndarray]", eps=3*np.finfo(float).eps):

I've filed this as a bug with the Python project.

As to why Bakuriu's suggestion of changing the expression to celestial_body.CelestialBody; that only 'works' because the name celestial_body raises a NameError exception. That name will never work in the context of your code, and thus the forward expression is not compliant with the PEP (it won't evaluate without errors once the module has been fully loaded).

If PyCharm accepts that reference anyway and correctly typechecks the function (e.g. it'll only let you use a numpy ndarray or CelestialObject instance when writing out a call), then that's due to PyCharm going beyond the specification here. Other tools are probably not going to be that forgiving.

In other words, as far as typing is concerned, you may as well have used frobnar.FlubberdyFlub as a forward reference there, it would have suppressed this specific bug just as much with the same effect; an invalid forward reference.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download