Andrey Reeshkov Andrey Reeshkov - 3 months ago 11
Python Question

Python: Do not map empty result of multiprocessing.Pool( )

Program have function that may return None value, for minimize spend time this function calling in parallel workers.
In code below, result of this function have the "None" values, how to exclude this is values from the "ret"?

#!/usr/bin/python
import sys,multiprocessing,time
maxNumber = sys.maxint+2

def numGen():
cnt=0
while cnt < maxNumber:
cnt +=1
yield cnt

def oddCheck(num):
global maxNumber
# time.sleep(1)
if not bool(num%1000000):
print "[%s%%] %s" % (int((num/maxNumber)*100),num)
if num%2:
return num

pool = multiprocessing.Pool( )
if sys.maxint < maxNumber:
print "We have problem on %s"%sys.maxint
# quit()
ret = pool.imap(oddCheck, numGen())
pool.close()
pool.join()

for x in ret:
print x

Answer

oddCheck returns None when num is not a even number (you do not have a return statement in this case, so None will be returned).

If you want to avoid None results, just use a list comprehension to filter them:

ret = [x for x in pool.imap(oddCheck, numGen()) if x is not None]

It should do the trick ;-)

Comments