Quite-0 Quite-0 - 7 months ago 239
Python Question

Python: Read all CSV files in directory

I know this has been asked before, but I've tried all the solutions and it's not quite working in my python code. I want to read ALL csv files in my directory: /root/Downloads and write some of the information to a new csv file called test.csv. So far, this code will only read and write from one csv file. Any ideas?

for currFile in os.listdir('/root/Downloads'):
if currFile.endswith('.csv'):
f1 = open('/root/Downloads/' + currFile)
c1 = open('/root/Downloads/test.csv', 'w')
r2 = f1.readlines()
w1 = csv.writer(c1)
for row in r2:
product = row.split(',')[1]
quantity = row.split(',')[2]
sales = row.split(',')[3]

w1.writerow([product, quantity, sales])

c1.close()
f1.close()


I've tried glob, os.walk, os.listdir and they aren't working. Perhaps I've been writing them with my code the wrong way.

Answer

You need to use glob, like this:

import glob
import csv

with open('results.csv', 'w') as f:
   writer = csv.writer(f)
   for filename in glob.iglob('/root/Downloads/*.csv'):
       with open(filename) as i:
           reader = csv.reader(i)
           for row in reader:
                writer.writerow([row[1], row[2], row[3]])