Shoeb Ahmed Shoeb Ahmed - 1 month ago 18
Python Question

How do I run a single function over a loop parallely? Python 2.7.12

I'm trying to parallelize a program which uses a single function in a for loop which updates a global list/variable parallely. How do I go about this and how can I pass values to the function?

The sample code looks like this,

#python 2.7.12
import random
sum=0

#sample function to take a random integer between 0 to i and add it to global variable sum
def hello(i):
print "Thread Id :",i #instead of i it should be the threadID
sum=sum+random.randrange(0,i)

#main function
i=1
for i in range(10):
hello(i) #how to parallelize this function over a loop?


Edit 1 :
Tried using the Process from multiprocessing but don't know how to pass values to the functions and how to run it parallely over a loop.

from multiprocessing import Process
sum=0

def Hello():
print 'hello: starting'
sum=sum+i #Don't know how to pass values here
print 'hello: finishing'

if name == 'main':
p1 = Process(target=func1)
p1.start()
p1.join()
print sum

Answer

You can use multiprocessing.dummy.Pool, which takes the function followed by your parameters (see below).

You'll also need to worry about synchronization on your global variable (see below for an example of how to use Lock).

Also, unless you use "global sum" the sum inside your function is referring to a local sum variable (see below for global sum example).

threading.current_thread() gives you the thread id.

#python 2.7.12
from multiprocessing.dummy import Pool as ThreadPool
import threading
import random

lock = threading.Lock()

sum = 0

#sample function to take a random integer between 0 to i and add it to global variable sum
def hello(i):
    if (i == 0):
        return
    global sum
    print threading.current_thread()   #instead of i it should be the threadID
    r = random.randrange(0,i)
    lock.acquire()
    try:
        sum = sum + r
    finally:
        lock.release()

ThreadPool().map(hello, list(range(1, 11)))

print sum
Comments