eviltnan eviltnan - 2 months ago 24
Python Question

using class methods as celery tasks

I'm trying to use the methods of class as the django-celery tasks, marking it up using @task decorator. The same situation is discribed here, asked by Anand Jeyahar.
It's something like this

class A:
@task
def foo(self, bar):
...

def main():
a = A()
...
# what i need
a.foo.delay(bar) # executes as celery task
a.foo(bar) # executes locally


The problem is even if i use class instance like this
a.foo.delay(bar)
it says, that
foo
needs at least two arguments, which meens that
self
pointer misses.

More information:


  • I can't convert class to module because of inheritance

  • Methods are strongly depended on class members, so i can't make them static

  • Marking class as the task with @task decorator makes the class a task itself, and it could be possible to execute the methods from
    run()
    method, using some argument as a key for method selection, but it's not exactly what i want.

  • Creating an instance of class and passing it as
    self
    argument to methods changes the way i execute the methods not as celery taks, but as usual methods (i.e. while testing)

  • I've tried to find out how i can register the task dinamically, from constructor for example, but celery shares the code between the workers, so that's why it seems to be impossible.



Thanks for your help!

Answer

Celery has experimental support for using methods as tasks since version 3.0.

The documentation for this is in celery.contrib.methods, and also mentions some caveats you should be aware of:

http://docs.celeryproject.org/en/latest/reference/celery.contrib.methods.html