Mary Mary - 3 months ago 10
Python Question

Exporting data to a CSV file, error can't encode character '\xe8'

I have a large set of data and I am going to export them into a CSV file using the following code.

with open ('/Users/mz/Dropbox/dis/Programming/zoloft.csv',
'w', newline = '') as zolo:
zolo = csv.writer(zolo, delimiter =',', quotechar='|')


rows = zip(all_rating, all_disorders, all_side_effects,
all_comments, all_gender, all_age, all_dosage_duration, all_date)

for row in rows:
zolo.writerow(row)


But there is the following error:

zolo.writerow(row)
UnicodeEncodeError: 'ascii' codec can't encode character '\xe8' in position 179: ordinal not in range(128)


Is there any way to handle this error inside the code I wrote? Thanks !

Answer

The csv module, at present, doesn't have Unicode support, but your dataset clearly contains Unicode characters. So what you can do is something similar to the answer to this question, and translate the Unicode characters to their nearest ASCII equivalents (so your text isn't illegible later).

I'd go with something like:

from unidecode import unidecode
with open('file', 'w', newline = '') as zolo:
    zolo = csv.writer(zolo, delimiter =',', quotechar='|')

    rows = zip(all_rating, all_disorders, all_side_effects,
               all_comments, all_gender, all_age,
               all_dosage_duration, all_date)

    for row in rows:       
        zolo.writerow(map(unidecode, row))