MasDaddy MasDaddy - 1 month ago 17
Python Question

Python: How do you stop numpy from multithreading?

I know this might seem like a ridiculous question, but I have to run jobs on a regular basis on compute servers that I share with others in the department and when I start 10 jobs, I really would like it to just take 10 cores and not more; I don't care if it takes a bit longer with a single core per run: I just don't want it to encroach on the others' territory, which would require me to renice the jobs and so on. I just want to have 10 solid cores and that's all.

More specifically, I am using Enthought 7.3-1 on Redhat, which is based on Python 2.7.3 and numpy 1.6.1, but the question is more general. I've been googling for some kind of an answer to this question for hours to no avail, so if someone knows of a switch in numpy that could turn off the multi-threading, please let me know.

Answer

Set the MKL_NUM_THREADS environment variable to 1. As you might have guessed, this environment variable controls the behavior of the Math Kernel Library which is included as part of Enthought's numpy build.

I just do this in my startup file, .bash_profile, with export MKL_NUM_THREADS=1. You should also be able to do it from inside your script to have it be process specific.