clg4 clg4 - 3 months ago 35
Python Question

Remove empty spaces or NaNs from lists in column of lists in Python/Pandas Dataframe

I have a Pandas dataframe

df
that looks like this:

A B
1 1 [a,b,d,d]
2 6 [,1,4,d,g]
3 a [w,1,NaN,x,y,2]


I need to remove the blank in row 2, and the NaN in row 3 to get:

A B
1 1 [a,b,d,d]
2 6 [1,4,d,g]
3 a [w,1,x,y,2]


I think applying some kind of lambda list comprehension?

df.B=df.B.apply(lambda x: x if x not in ['',np.NaN])


but not working...

Answer

You need to work on your comprehension skills :)

import numpy as np

df = pd.DataFrame([
        [1, ['a','b','d','d']],
        [6, ['',1,4,'d','g']],
        ['a', ['w',1,np.nan,'x','y',2]]
    ], columns=['A', 'B'])

df.B.apply(lambda l: [x for x in l if x not in ['', np.nan]])

where l is the current list and x are elements of l.