Don Don - 4 months ago 32
Python Question

Python - Multiprocessing Pool Class: Can the Thread Function contain arguments?

I want to know if it is possible to pass arguments into the threading pool funktion?

from multiprocessing.dummy import Pool as ThreadPool

def mainFunc():
myArray = ["A", "B", "C"]

pool = ThreadPool(8)
pool.map(threadFunc, myArray) # how is it possible to give >>threadFunc<< another argument >>text1<<
pool.close()
pool.join()

def threadFunc(text2):
text1 = "Hello " # this should be given as a argument of the function
print text1 + text2

mainFunc()


This is a simple example of what i want to do.. How can I give text1 as a second argument to the threadFunc function?

Solved:

I solved the problem with a simple global accessible variable ... but it was only a solution for the problem I had right now ... it is a greater idea to use the
multiprocessing.Process
instead ...

import sys
from multiprocessing.dummy import Pool as ThreadPool

this = sys.modules[__name__]

def mainFunc():
myArray = ["A", "B", "C"]
this.text1 = "Hello "

pool = ThreadPool(8)
pool.map(threadFunc, myArray) # how is it possible to give >>threadFunc<< another argument >>text1<<
pool.close()
pool.join()

def threadFunc(text2):
print this.text1 + text2

mainFunc()

Answer
  1. The official python docs says

    A parallel equivalent of the map() built-in function (it supports only one iterable argument though). It blocks until the result is ready.

    Therefore it is impossible to add another argument, but you can always add text1 as the last element of your myArray and pop it out once you use it.

  2. You can also use multiprocessing.Process other than pool. In this case you can add another argument easily.

    def f(name):
        print 'hello', name
    
    if __name__ == '__main__':
        p = Process(target=f, args=('bob',))
        p.start()
        p.join()
    
Comments