August Flanagan August Flanagan - 1 year ago 147
Python Question

Upload and parse csv file with google app engine

I'm wondering if anyone with a better understanding of python and gae can help me with this. I am uploading a csv file from a form to the gae datastore.

class CSVImport(webapp.RequestHandler):
def post(self):
csv_file = self.request.get('csv_import')
fileReader = csv.reader(csv_file)
for row in fileReader:

I'm running into the same problem that someone else mentions here -

That is, the csv.reader is iterating over each character and not the line. A google engineer left this explanation:

The call self.request.get('csv') returns a String. When you iterate over a
string, you iterate over the characters, not the lines. You can see the
difference here:

class ProcessUpload(webapp.RequestHandler):
def post(self):
file = open(os.path.join(os.path.dirname(__file__), 'sample.csv'))

# Iterating over a file
fileReader = csv.reader(file)
for row in fileReader:

# Iterating over a string
fileReader = csv.reader(self.request.get('csv'))
for row in fileReader:

I really don't follow the explanation, and was unsuccessful implementing it. Can anyone provide a clearer explanation of this and a proposed fix?


Answer Source

Short answer, try this:

fileReader = csv.reader(csv_file.split("\n"))

Long answer, consider the following:

for thing in stuff:
  print thing.strip().split(",")

If stuff is a file pointer, each thing is a line. If stuff is a list, each thing is an item. If stuff is a string, each thing is a character.

Iterating over the object returned by csv.reader is going to give you behavior similar to iterating over the object passed in, only with each item CSV-parsed. If you iterate over a string, you'll get a CSV-parsed version of each character.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download