Mehrdad Dadvand Mehrdad Dadvand - 3 months ago 17
Python Question

Processing a list whilst using threading

My code has a loop which checks a list and if there are items in the list it will process them and remove them from the list. The items are continuously inserted into the list by another thread.

Here is the code:

while True:
for item in list:
Process(item)
Remove(item)


Problem 1:

What I have above works but the problem is that it is very intensive on the cpu.

I can put something like 'time.sleep' in the code but this program needs to process the items in the list immediately without any delay. How do I make it less intensive on the cpu (remember that I am concurrently running another thread).

Problem 2:

I want to stop the loop from running when the list is empty and continue to process when something is added to the list. How do I go about doing this?

Answer

It seems that the queue module will be useful for you. It implements multi-producer, multi-consumer queues and allows synchronize between threads without busy waiting.

see also Solution to producer and consumer problem in python which uses this queue module.

Comments