user3543300 - 10 months ago 80

Python Question

I want to perform a simple operation, store a python dictionary into a csv file and then later read it from the file as a dictionary.

My dictionary maps a string to a list of numpy arrays, for example:

`d = {'x': [array([2, 3, 4]), array([5, 6, 7])], 'y': [array([1, 2, 3]), array([4, 5, 6])]`

So I would like to store this dictionary as a csv and then recreate the dictionary from the file in a different program.

I've tried using python's

`csv`

I also tried to use

`pandas`

`read_csv()`

Answer Source

```
In [610]: d = {'x': [np.array([2, 3, 4]), np.array([5, 6, 7])], 'y': [np.array(
...: [1, 2, 3]), np.array([4, 5, 6])]}
In [611]: d
Out[611]:
{'x': [array([2, 3, 4]), array([5, 6, 7])],
'y': [array([1, 2, 3]), array([4, 5, 6])]}
In [613]: np.save('test.npy',d)
In [614]: np.load('test.npy')
Out[614]: array({'x': [array([2, 3, 4]), array([5, 6, 7])], 'y': [array([1, 2, 3]), array([4, 5, 6])]}, dtype=object)
```

So calling `save`

on the dictionary, it wraps the dictionary in a object type array, and then saves that. And elements of an object array are saved with their respective pickle method. So it pickles a dictionary, and the lists within the dictionary. Finally the arrays within the lists pickled with a version of the `np.save`

. Lots of nesting. But it works.

And `item`

can be used to pull the dictionary out of the object array:

```
In [616]: dd=np.load('test.npy').item()
In [617]: dd
Out[617]:
{'x': [array([2, 3, 4]), array([5, 6, 7])],
'y': [array([1, 2, 3]), array([4, 5, 6])]}
```

And using `pickle`

directly:

```
In [626]: pickle.dump(d, open('test.pkl','wb'))
In [627]: np.load('test.pkl')
Out[627]:
{'x': [array([2, 3, 4]), array([5, 6, 7])],
'y': [array([1, 2, 3]), array([4, 5, 6])]}
In [629]: pickle.load(open('test.pkl','rb'))
Out[629]:
{'x': [array([2, 3, 4]), array([5, 6, 7])],
'y': [array([1, 2, 3]), array([4, 5, 6])]}
```

To write a `csv`

from this dictionary I'd create a structured array with fields named 'x' and 'y'. But I think I'd have to concatenate the arrays, so I can produce a 1d array. The csv would then look something like:

```
x y
2 1
3 4
....
```

Or if the subarrays are all the same length I might be able to produce

```
x y
2 5 1 4
3 6 2 5
...
```

In any if you still want to go the csv route, you need to decide how represent the values as simple lines of columns.