Kreender Kreender - 3 months ago 7
Python Question

Wait for all python processes to finish

I'm starting several python processes with downloads in a loop that calls this piece of code:

startTimeMillis = int(round(time.time() * 1000))

for i in range(10):
p = multiprocessing.Process(target=performCurl, args =("http://www.google.com/d", i, ))
p.start()


endTimeMillis = int(round(time.time() * 1000))
totalTimeSeconds = (endTimeMillis - startTimeMillis)
print "The whole process took ", str(totalTimeSeconds)


I want to check the time it takes for all the processes to finish, so how would I make the last part of the code to wait for all the processes?

Answer

Use p.join() to wait for a process to terminate

all_processes = [Process(...) for i in xrange(10)]
for p in all_processes:
  p.start()

for p in all_processes:
  p.join()