Tyler Crompton Tyler Crompton - 2 months ago 9
Python Question

How to import members of modules within a package

I am developing a package that has a file structure similar to the following:

test.py
package/
__init__.py
foo_module.py
example_module.py


If I call
import package
in test.py, I want the package module to appear similar to this:

>>> vars(package)
mapping_proxy({foo: <function foo at 0x…}, {example: <function example at 0x…})


In other words, I want the members of all modules in
package
to be in
package
's namespace, and I do not want the modules themselves to be in the namespace.
package
is not a sub-package.

Let's say my files look like this:

foo_module.py:

def foo(bar):
return bar


example_module.py:

def example(arg):
return foo(arg)


test.py:

print(example('derp'))


How do I structure the import statements in test.py, example_module.py, and __init__.py to work from outside the package directory (i.e. test.py) and within the package itself (i.e. foo_module.py and example_module.py)? Everything I try gives
Parent module '' not loaded, cannot perform relative import
or
ImportError: No module named 'module_name'
.

Also, as a side-note (as per PEP 8): "Relative imports for intra-package imports are highly discouraged. Always use the absolute package path for all imports. Even now that PEP 328 is fully implemented in Python 2.5, its style of explicit relative imports is actively discouraged; absolute imports are more portable and usually more readable."

I am using Python 3.3.

Answer

You said:

I want the members of all modules in package to be in package's namespace, and I do not want the modules themselves to be in the namespace.

I was able to do that by adapting something I've used in Python 2 to automatically import plug-ins to also work in Python 3.

In a nutshell, here's how it works: The package's __init__.py file imports all the other Python files in the same package directory that don't start with an '_' (underscore) character. After that, it then adds any names in the imported module's namespace to that of __init__ module's (which is also the package's namespace). Note I had to make the example_module module explicitly import foo from the foo_module.

One important aspect of doing things this way is realizing that it's dynamic and doesn't require the package module names to be hardcoded into the __init__.py file. Of course this requires more code to accomplish, but also makes it very generic and able to work with just about any (single-level) package -- because it will automatically import new modules when they're added and stop importing any removed from the directory.

test.py:

from package import *

print(example('derp'))

__init__.py:

def _import_all_modules():
    """dynamically imports all modules in the package"""
    import traceback
    import os
    global __all__
    __all__ = []
    globals_, locals_ = globals(), locals()

    # dynamically import all the package modules
    for filename in os.listdir(__name__):
        # process all python files in directory that don't start with underscore
        # (which also keeps this module from importing itself)
        if filename[0] != '_' and filename.split('.')[-1] in ('py', 'pyw'):
            modulename = filename.split('.')[0]  # filename without extension
            package_module = '.'.join([__name__, modulename])
            try:
                module = __import__(package_module, globals_, locals_, [modulename])
            except:
                traceback.print_exc()
                raise
            for name in module.__dict__:
                if not name.startswith('_'):
                    globals_[name] = module.__dict__[name]
                    __all__.append(name)

_import_all_modules()

foo_module.py:

def foo(bar):
    return bar

example_module.py:

from package.foo_module import foo  # added

def example(arg):
    return foo(arg)
Comments