Zhaitan - 1 year ago 71

Python Question

I have the following data, written in python 2, that I'd like to load to a python 3 file.

`import numpy as np`

x = np.array([{'a': np.array([1., 2., 3])}])

np.save('data.npy', x)

My first attempt was this:

`import numpy as np`

x = np.load('data.npy')

UnicodeError: Unpickling a python object failed

After playing around with the original data I am trying to load, it seems that whenever I have a numpy float inside of a numpy array inside a python dictionary, I get the error. I can load a dictionary, I can load a numpy array, I can even load a numpy array inside a python dictionary, but as soon as I have numpy floats inside a numpy array inside a python dictionary, I get the error. My second attempt was this:

`import numpy as np`

x = np.load('data.npy', encoding = 'bytes')

x

array([{b'a': array([ 1., 2., 3.])}], dtype=object)

This worked in that I could load the data, but it added a 'b' in front of every key in the dictionary. I was wondering if anyone had any insight into why this problem is occuring and how to fix it.

Thanks!

edit:

it seems like the following solved the problem:

`import numpy as np`

x = np.load('data.npy', encoding = 'latin1')

I'm not quite sure why, but it seemed to work.

Answer Source

The default encoding in Python 2 is `ascii`

; in Python 3 it is `utf-8`

. `latin1`

(a.k.a., ISO-8859-1) is a superset of `ascii`

. That's why loading `ascii`

-encoded strings with `latin1`

works and gives the same result as loading it with `ascii`

.