valerio92 valerio92 - 1 year ago 89
Python Question

Python: run script in all subdirectories

I am new to Python and I am using it to do some data analysis.

My problem is the following: I have a directory with many subdirectories, each one of which contains a large number of data files.

I already wrote a Python script which, when executed in one of those subdirectories, performs the data analysis and writes it on a output file. The script includes some shell commands that I call using

, so I have to "be" in one of the subdirectories for it to work.

How can I write a function that automatically:

  1. Moves into the first subdirectory

  2. Executes the script

  3. Goes back to the parent directory and moves to the next subdirectory

I guess that this could be done in some way using
but I didn't really understand how it works.

PS I am aware of the existence of this post but it doesn't solve my problem.

PPS Maybe I should point out that my function does not take the directory name as argument. Actually it takes no argument.

Answer Source

To change your working directory in Python you need:


You can then recursively run your script.

Example Code:

import os

directory_to_check = "your_dir" # Which directory do you want to start with?

def my_function(directory):
      print("Listing: " + directory)
      print("\t-" + "\n\t-".join(os.listdir("."))) # List current working directory

# Get all the subdirectories of directory_to_check recursively and store them in a list:
directories = [os.path.abspath(x[0]) for x in os.walk(directory_to_check)]
directories.remove(os.path.abspath(directory_to_check)) # If you don't want your main directory included

for i in directories:
      os.chdir(i)         # Change working Directory
      my_function(i)      # Run your function

I don't know how your script works because your question is quite general, so therefore I can only give a general answer....

But I think what you need is:

  1. Get all subdirectories and store them using os.walk
  2. Change your working directory with os.chdir

os.walk alone won't work

I hope this helps! Good luck!