nml nml - 1 month ago 55
Python Question

ImportError: cannot import name celery

I'm trying to run some background jobs using Celery + Redis + Flask.

My app structure is:

myapp/
celery_worker.py
manage.py
myapp/
__init__.py
app.py
bot/
__init__.py
tasks.py
accounts/
views.py


I initialise celery into app.py as:

celery = Celery('tasks', include=['bot.tasks'])

def create_app(config=None, app_name=None, blueprints=None):
# more Celery and other config here like celery.conf.update()


And start worker by
celery_worker.py


from myapp import create_app, celery

app = create_app()
app.app_context().push()


tasks.py
contains

from myapp import celery

@celery.task(name='my_task_bot')
def my_task_bot():
# pass


Now, when I try to access tasks from tasks.py into accounts/views.py with following import:

from ..bot.tasks import my_task_bot


I getting
ImportError: cannot import name celery
error.

No matter, where I try to import tasks, I'm getting this error. This
celery
instance seems to be the one initialised in app.py, but is not getting imported.

PS: I'm following pattern of https://github.com/ezequielo/flask_celery_exp repo and it works fine. But not my app.

Edit:

Here is Traceback:

Traceback (most recent call last):
File "manage.py", line 8, in <module>
from myapp import create_app
File "/home/mars/myapp/myapp/__init__.py", line 3, in <module>
from app import create_app, celery
File "/home/mars/myapp/myapp/app.py", line 18, in <module>
from .accounts import (accounts, AccountsAdmin)
File "/home/mars/myapp/myapp/accounts/__init__.py", line 7, in <module>
from .views import accounts
File "/home/mars/myapp/myapp/accounts/views.py", line 6, in <module>
from ..bot.tasks import my_task_bot
File "/home/mars/myapp/myapp/bot/tasks.py", line 14, in <module>
from myapp import celery
ImportError: cannot import name celery

Fatal error: local() encountered an error (return code 1) while executing 'python manage.py initdb'


I get above error on trying to start any of either
manage.py
or
celery_worker
.

Please suggest.

Answer

Your import logic is not correct and leads to circular dependency. Do not initialize celery in app.py together with reference to the module (accounts) that in turn imports celery instance.

You have several options:

  1. Remove dependency from app accounts at app.py module

  2. Put celery initialization object into shared module that does not refer to any other modules, but referred by /*/tasks.py submodules.

  3. Do not import celery in tasks, but use shared_task decorator. For more info refer to doc