it_is_a_literature it_is_a_literature - 5 months ago 49
Python Question

how to pass argument into threading?

I want to add 5 for every element in range(1,100) with threading module,
to watch which rusult is in which thread.
I finished almost of the code,but how to pass argument into threading.Thread?

import threading,queue
x=range(1,100)
y=queue.Queue()
for i in x:
y.put(i)

def myadd(x):
print(x+5)


for i in range(5):
print(threading.Thread.getName())
threading.Thread(target=myadd,args=x).start() #it is wrong here
y.join()


Thinks to dano ,it is ok now ,in order to run in interactive way, i rewrite it as:

method 1:run in interactive way.

from concurrent.futures import ThreadPoolExecutor
import threading
x = range(1, 100)

def myadd(x):
print("Current thread: {}. Result: {}.".format(threading.current_thread(), x+5))

def run():
t = ThreadPoolExecutor(max_workers=5)
t.map(myadd, x)
t.shutdown()
run()


methdo 2:

from concurrent.futures import ThreadPoolExecutor
import threading
x = range(1, 100)
def myadd(x):
print("Current thread: {}. Result: {}.".format(threading.current_thread(), x+5))
def run():
t = ThreadPoolExecutor(max_workers=5)
t.map(myadd, x)
t.shutdown()
if __name__=="__main__":
run()


What about if more args to be passed into ThreadPoolExecutor?
I want to calculate 1+3, 2+4, 3+45 until 100+102 with multi-processing module.
And what about 20+1,20+2,20+3 until 20+100 with multi-processing module?

from multiprocessing.pool import ThreadPool
do = ThreadPool(5)
def myadd(x,y):
print(x+y)

do.apply(myadd,range(3,102),range(1,100))


How to fix it?

Answer

It looks like you're trying to create a thread pool manually, so that five threads are used to add up all 100 results. If this is the case, I would recommend using multiprocessing.pool.ThreadPool for this:

from multiprocessing.pool import ThreadPool
import threading
import queue

x = range(1, 100)

def myadd(x):
    print("Current thread: {}. Result: {}.".format(
               threading.current_thread(), x+5))

t = ThreadPool(5)
t.map(myadd, x)
t.close()
t.join()

If you're using Python 3.x, you could use concurrent.futures.ThreadPoolExecutor instead:

from concurrent.futures import ThreadPoolExecutor
import threading

x = range(1, 100)

def myadd(x):
    print("Current thread: {}. Result: {}.".format(threading.current_thread(), x+5))

t = ThreadPoolExecutor(max_workers=5)
t.map(myadd, x)
t.shutdown()

I think there are two issues with your original code. First, you need to pass a tuple to the args keyword argument, not a single element:

threading.Thread(target=myadd,args=(x,))

However, you're also trying to pass the entire list (or range object, if using Python 3.x) returned by range(1,100) to myadd, which isn't really what you want to do. It's also not clear what you're using the queue for. Maybe you meant to pass that to myadd?

One final note: Python uses a Global Interpreter Lock (GIL), which prevents more than one thread from using the CPU at a time. This means that doing CPU-bound operations (like addition) in threads provides no performance boost in Python, since only one of the threads will ever run at a time. Therefore, In Python it's preferred to use the multiple processes to parallelize CPU-bound operations. You could make the above code use multiple processes by replacing the ThreadPool in the first example with from mulitprocessing import Pool. In the second example, you would use ProcessPoolExecutor instead of ThreadPoolExecutor. You would also probably want to replace threading.current_thread() with os.getpid().

Edit:

Here's how to handle the case where there are two different args to pass:

from multiprocessing.pool import ThreadPool

def myadd(x,y):
    print(x+y)

def do_myadd(x_and_y):
    return myadd(*x_and_y)

do = ThreadPool(5)    
do.map(do_myadd, zip(range(3, 102), range(1, 100)))

We use zip to create a list where we pair together each variable in the range:

[(3, 1), (4, 2), (5, 3), ...]

We use map to call do_myadd with each tuple in that list, and do_myadd uses tuple expansion (*x_and_y), to expand the tuple into two separate arguments, which get passed to myadd.