The Spiteful Octopus The Spiteful Octopus - 3 months ago 13
Python Question

running script multiple times simultaniously in python 2.7

Hello I am trying to run a script multiple times but would like this to take place at the same time from what I understood i was to use subprocess and threading together
however when i run it it still looks like it is being executed sequentially can someone help me so that i can get it to run the same script over and over but at the same time?
is it in fact working and just really slow?

edit forgot the last piece of code now at the bottom

here is what i have so far

import os
import datetime
import threading
from subprocess import Popen

today = datetime.date.today()
os.makedirs("C:/newscript_image/" + str(today))

class myThread(threading.Thread):
def run(self):
for filename in os.listdir('./newscript/'):
if '.htm' in filename:
name = filename.strip('.htm')

dbfolder = "C:/newscript/db/" + name
os.makedirs(dbfolder)

Popen("python.exe C:/execution.py" + ' ' + filename + ' ' + name + ' ' + str(today) + ' ' + dbfolder)
myThread().start()

Answer

Personally, I'd use multiprocessing. I'd write a function that takes a filename and does whatever the main guts of execution does (probably by importing execution and running some function within it):

import multiprocessing
import execution
import datetime

#assume we have a function:
#exection.run_main_with_args(filename,name,today_str,dbfolder)

today = datetime.datetime.today()
def my_execute(filename):
    if '.htm' in filename:
       name = filename.strip('.htm')
       dbfolder = "C:/newscript/db/" + name
       os.makedirs(dbfolder)
       execution.run_main_with_args(filename,name,str(today),dbfolder)

p = multiprocessing.Pool()
p.map(my_execute,list_of_files_to_process)