LearningSlowly LearningSlowly - 3 months ago 15
Python Question

CSV read error: new-line character seen in unquoted field

I created a python script which works with a test CSV dataset of 10 records. When I scaled this up to the actual datasets (a few thousand rows), I am getting the following error:


_csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?


The code is as follows:

with open('./Origins.csv', 'r') as csvfile:
reader = csv.DictReader(csvfile)
origincoords = ['{Y},{X}'.format(**row) for row in reader]


The full error code is:

Traceback (most recent call last):
File "./Driving.py", line 14, in <module>
origincoords = ['{Y},{X}'.format(**row) for row in reader]
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/csv.py", line 103, in next
self.fieldnames
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/csv.py", line 90, in fieldnames
self._fieldnames = self.reader.next()
_csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?


Perhaps there is a scale problem with the CSV reading method I am using?

Answer

From PEP-0278:

In a Python with universal newline support open() the mode parameter can also be "U", meaning "open for input as a text file with universal newline interpretation". Mode "rU" is also allowed, for symmetry with "rb"

So try to change

with open('./Destinations.csv', 'r') as csvfile:

to

with open('./Destinations.csv', 'rb') as csvfile:

If the error persists, change to

with open('./Destinations.csv', 'rU') as csvfile:

Edited accorded to Martijn Pieters's comment.