J_Heads J_Heads - 2 months ago 5
Python Question

Convert columns from a data frame to a list of dicts efficiently

I am converting columns of data frame to to a list of dictionaries, however, due to the number of columns and number of observations in my data frame I run out of memory using my current approach:

df = pd.DataFrame(np.random.randn(10, 3), columns=['a', 'b', 'c'])
df.T.to_dict().values()


Is there a more efficient way I can do this?

Answer

is that what you want?

In [9]: df.to_dict('r')
Out[9]:
[{'a': 1.3720225964856179,
  'b': -1.1530341240730422,
  'c': -0.18791193632296455},
 {'a': 1.3283240103713496, 'b': 3.6614598433626959, 'c': -0.46395170547460196},
 {'a': -1.4960282310010959,
  'b': 0.25156344524211743,
  'c': -1.3664311385849288},
 {'a': -0.11601714495988308,
  'b': -0.73400546410732148,
  'c': 0.9131316189984563},
 {'a': 0.27404065198912386,
  'b': -3.1246509560345261,
  'c': 0.67227710572588184},
 {'a': 1.3390654954886572, 'b': -0.80535280826120292, 'c': -1.78092490531724},
 {'a': -0.13911682611874573,
  'b': 1.6846890792762916,
  'c': 0.22985191293512194},
 {'a': -0.22058925847227495,
  'b': -0.29342906413451442,
  'c': -1.1181888670510167},
 {'a': 3.2190577575509951, 'b': 0.59152576294942738, 'c': -1.3474566325216308},
 {'a': -0.53486658456919434, 'b': 0.14390073779727405, 'c': 1.2214292373636}]

data:

In [10]: df
Out[10]:
          a         b         c
0  1.372023 -1.153034 -0.187912
1  1.328324  3.661460 -0.463952
2 -1.496028  0.251563 -1.366431
3 -0.116017 -0.734005  0.913132
4  0.274041 -3.124651  0.672277
5  1.339065 -0.805353 -1.780925
6 -0.139117  1.684689  0.229852
7 -0.220589 -0.293429 -1.118189
8  3.219058  0.591526 -1.347457
9 -0.534867  0.143901  1.221429
Comments