Clíodhna Clíodhna - 1 year ago 69
Python Question

Loop that will iterate a certain number of times through a CSV in Python

I have a large CSV file (~250000 rows) and before I work on fully parsing and sorting it I was trying to display only a part of it by writing it to a text file.

csvfile = open(file_path, "rb")
rows = csvfile.readlines()
text_file = open("output.txt", "w")
row_num = 0
while row_num < 20:
text_file.write(", ".join(row[row_num]))
row_num += 1

I want to iterate through the CSV file and write only a small section of it to a text file so I can look at how it does this and see if it would be of any use to me. Currently the text file ends up empty.

A way I thought might do this would be to iterate through the file with a for loop that exits after a certain number of iteration but I could be wrong and I'm not sure how to do this, any ideas?

Answer Source

There's nothing specifically wrong with what you're doing, but it's not particularly Pythonic. In particular reading the whole file into memory with readlines() at the start seems pointless if you're only using 20 lines.

Instead you could use a for loop with enumerate and break when necessary.

csvfile = open(file_path, "rb")
text_file = open("output.txt", "w")
for i, row in enumerate(csvfile):
    if row_num >= 20:

You could further improve this by using with blocks to open the files, rather than closing them explicitly. For example:

with csvfile as open(file_path, "rb"):
    #your code here involving csvfile
#now the csvfile is closed!

Also note that Python might not be the best tool for this - you could do it directly from Bash, for example, with just head -n20 csvfile.csv > output.txt.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download