cᴏʟᴅsᴘᴇᴇᴅ cᴏʟᴅsᴘᴇᴇᴅ - 2 months ago 9
Python Question

Replacing NaNs with mutable types in a pandas dataframe

In the process of trying to solve another question, I've hit a roadblock. Here's my data:

col1 col2 col3 col4 col5 user_ID
0 [1] [3] [] NaN NaN 1
1 [2, 3] [3] [1, 2, 3] NaN NaN 2
2 [3] [3, 1] [3, 1] NaN NaN 3
0 [1, 2] NaN [1] [3] NaN 1
1 [3] NaN [2, 3] [3] NaN 2
2 [3] NaN [3] [3, 1] NaN 3
0 [1] [3] NaN NaN [] 1
1 [2, 3] [3] NaN NaN [1, 2, 3] 2
2 [3] [3, 1] NaN NaN [3, 1] 3


I want to replace those NaNs with an empty list so I can perform a summation along those columns.

I've tried
df.replace
, but I get

TypeError: Invalid "to_replace" type: 'float'


I also tried
df.fillna
and got

TypeError: "value" parameter must be a scalar or dict, but you passed a "list"


How can I fill these NaNs with the empty list
[]
?




Edit: So it turns out this is a duplicate! Since the marked dupe doesn't have an
applymap
solution, I'll keep this here.

Answer Source

A little dirty, but I solved this using df.applymap:

In [671]: df.applymap(lambda x: [] if x != x else x)
Out[671]
     col1    col2       col3    col4       col5  user_ID
0     [1]     [3]         []      []         []        1
1  [2, 3]     [3]  [1, 2, 3]      []         []        2
2     [3]  [3, 1]     [3, 1]      []         []        3
0  [1, 2]      []        [1]     [3]         []        1
1     [3]      []     [2, 3]     [3]         []        2
2     [3]      []        [3]  [3, 1]         []        3
0     [1]     [3]         []      []         []        1
1  [2, 3]     [3]         []      []  [1, 2, 3]        2
2     [3]  [3, 1]         []      []     [3, 1]        3