warvariuc warvariuc - 4 months ago 16
Python Question

Import paths - the right way?

I know there are A LOT of similar or the same questions, but i still cannot understand / find the right way for me to work with modules. Python is my favorite language, and i like everything in it except working with imports: recursive imports (when you try to reference a name that is not yet there), import paths, etc.

So, I have this kind of a project structure:

my_project/
package1/
__init__.py
module1
module2
package2/
__init__.py
module1
module2


Package1
may be used a standalone unit, but is also expected to be imported by
package2
.
What am i doing now, is that, for example, in
package1.module1
i write
from package1 import module2
, i.e. using full path to imported module. I do this because if i use
import module2
-- this will not work when the module will be imported from another package (
package2
). I also cannot use
from . import module2
-- this will not work when running
module1
directly.

OK, so for
from package1 import module2
in
package1.module1
to work in both cases (when running directly
package1.module1
and when importing it from
package2
) i add these lines at the beginning of
package1.module1
:

import os, sys
currDir = os.path.dirname(os.path.realpath(__file__))
rootDir = os.path.abspath(os.path.join(currDir, '..'))
if rootDir not in sys.path: # add parent dir to paths
sys.path.append(rootDir)


For me this works, but i feel this is not pythonic. Am i doing something wrong?

Should i, instead, always run
package1.module1
from project root? If so, this makes inconvenient to run it from an IDE -- i need somehow to set paths in it.

UPDATE: I tried to add a file
root.pth
to
package1
dir with contents of
..
. But it didn't work -- i guess it's intended for something else.

CONCLUSIONS:


  1. Always use absolute imports:
    import package1.module1

  2. Add a bootstrapper to the root folder to start some of the modules as a standalone script. This solves running the script form an IDE and is a pythonic approach.



On 4/22/07, Brett Cannon wrote:


This PEP is to change the
if __name__ == "__main__": ...
idiom to
if __name__ == sys.main: ...
so that you at least have a chance
to execute module in a package that use relative imports.

Ran this PEP past python-ideas. Stopped the discussion there when too
many new ideas were being proposed. =) I have listed all of them in
the Rejected Ideas section, although if overwhelming support for one
comes forward the PEP can shift to one of them.


I'm -1 on this and on any other proposed twiddlings of the
__main__

machinery. The only use case seems to be running scripts that happen
to be living inside a module's directory, which I've always seen as an
antipattern. To make me change my mind you'd have to convince me that
it isn't.

--Guido van Rossum

Answer

What is the entry point for your program? Usually the entry point for a program will be at the root of the project. Since it is at the root, all the modules within the root will be importable, provided there is an __init__.py file in them.

So, using your example:

my_project/
    main.py
    package1/
        __init__.py
        module1
        module2
    package2/
        __init__.py
        module1
        module2

main.py would be the entry point for your program. Because the file that is executed as main is automatically put on the PYTHONPATH, both package1 and package2 are available from the top level import.

# in main.py
from package1.module1 import *
from package1.module2 import *

# in package1.module1
import module2
from package2.module1 import *

# in package2.module1 import *
import module2
from package1.module1 import *

Note that in the above, package1 and package2 depend on each other. That should never be the case. But this is just an example of being able to import from anywhere.

main.py doesn't have to be anything fancy either. It can be very simple:

# main.py

if __name__ == '__main__':
    from package1.module1 import SomeClass
    SomeClass().start()

The point I'm trying to make, is that if a module needs to be accessible by other modules, that module should be available as a top level import. A module should not attempt to put itself as a top level import (directly on the PYTHONPATH).

It should be the responsibility of the project for ensuring that all imports can be satisfied if the module is included directly in the project. There are two ways to do this. The first is by creating a bootstrapper file such as main.py in the project folder. The other, is by creating a file that adds all relevant paths to PYTHONPATH, that is loaded by any entry points that may exist.

For example:

# setup.py
import sys

def load():
    paths = ['/path1/','/path2/','/path3/']
    for p in path:
        sys.path.insert(0, p)

# entrypoint.py
from setup import load
load()
# continue with program

The main thing to take away, is that a module is not supposed to put itself on the path. The path should be determined automatically by the entry point into the program, or defined explicitly by a setup script that knows where all the relevant modules are.

Comments