PascalvKooten PascalvKooten - 3 months ago 11
Python Question

Importing module with "current-directory" imports

I have the following code to load a module dynamically:

def load_module(absolute_path):
import importlib.util
module_name, _ = os.path.splitext(os.path.split(absolute_path)[-1])
try:
py_mod = imp.load_source(module_name, absolute_path)
except ImportError:
module_root = os.path.dirname(absolute_path)
print("Could not directly load module, including dir: {}".format(module_root))
spec = importlib.util.spec_from_file_location(
module_name, absolute_path, submodule_search_locations=[module_root, "."])
py_mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(py_mod)
return py_mod


It works really well, except for the case when it is trying to import a script in the same folder (and not part of a package with the same name). For example, script
a.py
is doing
import b
. It results in the error
ImportError: No module named 'b'
(which is common for Python 3).

But I would really like to find a way to solve this? It would be solved by prepending:

import sys
sys.path.append(".")


to script "a".

Though I hoped it could have been solved by:

submodule_search_locations=[module_root, "."]


Oh yea, the rationale is that I also want to support importing modules that are not proper packages/modules, but just some scripts that would work in an interpreter.

Reproducible code:

~/example/a.py



import b


~/example/b.py



print("hi")


~/somewhere_else/main.py (located different from a/b)



import sys, os, imp, importlib.util

def load_module(absolute_path) ...

load_module(sys.argv[1])


Then run on command line:

cd ~/somewhere_else
python3.5 main.py /home/me/example/a.py


which results in
ImportError: No module named 'b'


The following code solves it, but of course we cannot put
sys.path
stuff manually in all scripts.

~/example/a.py (2)



import sys
sys.path.append(".")
import b


I really hope others might have a solution that I didn't think of yet.

Appendix



def load_module(absolute_path):
import importlib.util
module_name, _ = os.path.splitext(os.path.split(absolute_path)[-1])
try:
py_mod = imp.load_source(module_name, absolute_path)
except ImportError as e:
if "No module named" not in e.msg:
raise e

missing_module = e.name
module_root = os.path.dirname(absolute_path)

if missing_module + ".py" not in os.listdir(module_root):
msg = "Could not find '{}' in '{}'"
raise ImportError(msg.format(missing_module, module_root))

print("Could not directly load module, including dir: {}".format(module_root))
sys.path.append(module_root)
spec = importlib.util.spec_from_file_location(module_name, absolute_path)
py_mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(py_mod)
return py_mod

Answer

It doesn't really matter that this you are using dynamic importing here; the same problem applies to any code making assumptions about the current directory being on the path. It is the responsibility of those scripts to ensure that the current directory is on the path themselves.

Rather than use '.' (current working directory), use the __file__ global to add the directory to the path:

import os.path
import sys

HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, HERE)

You could retry your dynamic import by adding os.path.dirname(absolute_path) to sys.path when you have an ImportError (perhaps detecting it was a transient import that failed), but that's a huge leap to make as you can't distinguish between a missing dependency and a module making an assumption about sys.path.

Comments