djma djma - 1 year ago 89
Python Question

How to structure a python project with ipython notebooks?

After looking at many answers here, I still dont understand how to structure a python project with imports...

Let's say my dir structure is

myproject
utils
tool1.py
datasets
create_bla_dataset.py
research
mynote.ipynb


now I import tool1 from create_bla_dataset.py with

# We are in create_bla_dataset.py
from ..utils import tool1

... define how to create a dataset

if __name__ == "__main__":
# demo the dataset


and I can run the file with

> cd <dir_contains_myproject>
> python -m myproject.datasets.create_bla_dataset


But now I want to import create_bla_dataset inside mynote.ipynb while opening mynote.ipynb with a jupyter notebook, but I have no idea how...

If I try
from ..datasets import create_bla_dataset
, I get
ValueError: attempted relative import beyond top-level package


edit: I tried launching jupyter notebook from various directories but no luck, same error.

Answer Source

When you use IPython/Jupyter Notebooks you need take account that ipython server only see files from a root directory defined from where you launch the notebook instance. In this case, maybe you are launched directly the notebook file and not from the project root.

Also, you can use __init__.py files.

Updated: You can add a cell at top of the notebook.

import sys, os
sys.path.insert(0, os.path.dirname(os.path.abspath('..')))

Then, myproject is recognized as a package and you can import as is.

from myproject.datasets.create_bla_dataset import bar
bar()
>>> 'hi foo hi bar'

This is because relative import only work in packages and if you need use as package you need to add the path where is the package.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download