fehu fehu - 17 days ago 11
Python Question

How to avoid Django sub thread being killed by uWSGI respawn

I created a web server by Django + uWSGI. The basic flow is: when a request is received, Django will initiated a sub thread by python build-in lib "Threading" to write db asynchronously, and in main thread it will respond immediately back to client.

How, uWSGI sometimes will respawn the worker process(maybe when there is no request handling by the process?), which caused the background sub thread is also killed even when it's not finished yet. Any clue to avoid uWSGI not to respawn a worker process which has a running sub-thread?

uWSGI respawn log:

DAMN ! worker 4 (pid: 31161) died, killed by signal 9 :( trying respawn ...


uWSGI ini config (version 2.0.12):

[uwsgi]
# Django's wsgi file
wsgi-file = /home/fh/dj_uwsgi/dj_site/dj_site/wsgi.py
master = true
processes = 10
http = :8001
threads = 2
enable-threads = true
http-timeout = 10
max-requests = 5000
chmod-socket = 664
vacuum = true
pidfile = /home/fh/dj_uwsgi/dj_site/uwsgi.pid
daemonize = /home/fh/log/uwsgi_dj.log


Django(version 1.8) app pseudo code:

in handlers.py:

import threading

class SubThreadClass(threading.Thread):
daemon = True

def run(self):
# actions to write db

def myHandler():
my_sub_thread = SubThreadClass()
my_sub_thread.start()


in views.py:

from handlers import myHandler

def url_handler(request):
myHandler()
return HttpResponse()

Answer

Manually dealing with threads is rarely a good idea in an environment you do not manage.

You should use an offline worker system like Celery.

Comments