Eric Sauer Eric Sauer - 10 months ago 76
Python Question

Multithreading performance overhead

So basically I created this program that adds values to redis. So far I get this timing:

real 0m27.759s
user 0m18.129s
sys 0m5.580s

However when I tried to run multiple threads:

if __name__ == '__main__':
for x in range(0, NUM_THREADS):
Thread(None, startProgram, None,
except Exception as errtxt:
print errtxt

I get this with
set ot

real 0m32.642s
user 0m22.953s
sys 0m11.473s

Why is my program running slower with more threads?

I'm running Linux Ubuntu 11.04 and Python 2.7.1.

Answer Source

The result depends on the Python implementation, cpython's GIL prevents parallel computations from being faster than sequential ones.

Consider using the multiprocessing module, which executes each thread in its own Python process, or alternative GIL-free Python implementations like IronPython and Jython.