porton porton - 2 months ago 20
Python Question

Implementing a single thread server/daemon (Python)

I am developing a server (daemon).

The server has one "worker thread". The worker thread runs a queue of commands. When the queue is empty, the worker thread is paused (but does not exit, because it should preserve certain state in memory). To have exactly one copy of the state in memory, I need to run all time exactly one (not several and not zero) worker thread.

Requests are added to the end of this queue when a client connects to a Unix socket and sends a command.

After the command is issued, it is added to the queue of commands of the worker thread. After it is added to the queue, the server replies something like "OK". There should be not a long pause between server receiving a command and it "OK" reply. However, running commands in the queue may take some time.

The main "work" of the worker thread is split into small (taking relatively little time) chunks. Between chunks, the worker thread inspects ("eats" and empties) the queue and continues to work based on the data extracted from the queue.

How to implement this server/daemon in Python?

Answer

This is a sample code with internet sockets, easily replaced with unix domain sockets. It takes whatever you write to the socket, passes it as a "command" to worker, responds OK as soon as it has queued the command. The single worker simulates a lengthy task with sleep(30). You can queue as many tasks as you want, receive OK immediately and every 30 seconds, your worker prints a command from the queue.

import Queue, threading, socket
from time import sleep

class worker(threading.Thread):
    def __init__(self,q):
        super(worker,self).__init__()
        self.qu = q

    def run(self):
        while True:
            new_task=self.qu.get(True)
            print new_task
            sleep(30)

task_queue = Queue.Queue()
w = worker(task_queue)
w.daemon = True
w.start()

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind(('localhost', 4200))
sock.listen(1)
try:
    while True:
        conn, addr = sock.accept()
        data = conn.recv(32)
        task_queue.put(data)
        conn.sendall("OK")
        conn.close()
except:
    sock.close()