user6745741 user6745741 - 16 days ago 5
Python Question

How to parallelize functions that return values in python?

I have two functions,core_ensembling and sampling.core ensembling return values.I need to parallelize the core_ensebling function inside sampling function.that is i need to parallely compute set1,set2,set3...set6.All this computations are independent.So is it possible to parallelize this program?.How can i do this?i am using windows 10,intel i3 processor which has 2 cores and python 2.7

def core_ensembling(args):
#some code#
return ensemble_set

def sampling(p,q):
total_frames=q
ensemble_row=6
ensemble_col=8
frame_ensemble=[]
ensemble_set=[]

args1=[8,0,80,total_frames,p]
args2=[16,80,160,total_frames,p]
args3=[24,160,240,total_frames,p]
args4=[32,240,320,total_frames,p]
args5=[40,320,400,total_frames,p]
args6=[48,400,480,total_frames,p]

# parallising part

set1=core_ensembling(args1)
set2=core_ensembling(args2)
set3=core_ensembling(args3)
set4=core_ensembling(args4)
set5=core_ensembling(args5)
set6=core_ensembling(args6)
ensemble = list(itertools.chain(set1,set2,set3,set4,set5,set6))
return ensemble

Answer
import multiprocessing
pool = multiprocessing.Pool()
set1 = pool.apply_async(core_ensembling, args1)
# ...
set6 = pool.apply_async(core_ensembling, args6)
ensemble = list(itertools.chain(set1.get(),set2.get(),set3.get(),set4.get(),set5.get(),set6.get()))
Comments