wierob wierob - 3 months ago 20
Python Question

Python 2 CSV writer produces wrong line terminator on Windows

According to the its documentation csv.writer should use '\r\n' as lineterminator by default.

import csv

with open("test.csv", "w") as f:
writer = csv.writer(f)

rows = [(0,1,2,3,4),
(-0,-1,-2,-3,-4),
("a","b","c","d","e"),
("A","B","C","D","E")]

print writer.dialect.lineterminator.replace("\r", "\\r").replace("\n", "\\n")
writer.writerows(rows)
print writer.dialect.lineterminator.replace("\r", "\\r").replace("\n", "\\n")


This prints

\r\n
\r\n


as expected. But, the created csv-file uses the lineterminator '\r\r\n'

0,1,2,3,4

0,-1,-2,-3,-4

a,b,c,d,e

A,B,C,D,E


Is this a bug or is there something wrong in my usage of csv.writer?

Python version:


ActivePython 2.6.2.2 (ActiveState
Software Inc.) based on Python 2.6.2
(r262:71600, Apr 21 2009, 15:05:37)
[MSC v.1500 32 bit (Intel)] on win32


on Windows Vista

Answer

In Python 2.x, always open your file in binary mode, as documented. csv writes \r\n as you expected, but then the underlying Windows text file mechanism cuts in and changes that \n to \r\n ... total effect: \r\r\n

From the csv.writer documentation:

If csvfile is a file object, it must be opened with the 'b' flag on platforms where that makes a difference.

There seems to be some reticence about actually uttering the name of the main culprit :-)

Comments