monojohnny monojohnny - 4 months ago 28
Python Question

How to get CSV reader to read a memory file?

I'm trying to fetch an entire file into memory (done - using StringIO) - but these objects don't really behave exactly like 'real' files as far as I can see - I get the whole contents, or I can read a line at a time, but I can't work out how to apply this pattern:

import csv
with open(#MYMEMORYFILE_HERE#, 'rb') as csvfile:
spamreader = csv.reader(csvfile, delimiter=' ', quotechar='|')
for row in spamreader:


Is there a way of treating a memoryfile just like an on-disk file , so I can use the nicer idioms above ?

Python 2.7.4 (default, Apr 19 2013, 18:28:01)


EDIT: Thanks for the answers on this - I think I have narrowed down what was confusing me...but I still have an issue here, the following doesn't output anything? I suspect flushing ?

from csv import reader, writer
import StringIO

memfile=StringIO.StringIO()
spamwriter = writer(memfile)
spamwriter.writerow(['Spam'] * 5 + ['Baked Beans'])
spamwriter.writerow(['Spam', 'Lovely Spam', 'Wonderful Spam'])
spamreader=reader(memfile)
for row in spamreader:
print ', '.join(row)
memfile.close()


EDIT#2: But I'm barking up the wrong tree I think: I couldn't get the on-disk version of this to work either ('IOError: File not open for reading' - when I call the read on the already open file...)
EDIT#3: abandoned the StringIO (no real need for it) - used splitlines as per the answer.

I'll leave the code and comments here - in case it's useful. (even though it is a cul-de-sac).

llb llb
Answer

If you want to read a whole file into memory, and it's text, you don't need to open a StringIO object. Just read it as a string!

with open(filename, 'r') as file:
    in_memory_file = file.read()

Then you can use splitlines to iterate over it the way you would iterate over an open text file.

spamreader = csv.reader(in_memory_file.splitlines(), delimiter=' ', quotechar='|')
for row in spamreader:
    pass