clg4 clg4 - 4 months ago 45
Python Question

Remove empty spaces or NaNs from lists in column of lists in Python/Pandas Dataframe

I have a Pandas dataframe

that looks like this:

1 1 [a,b,d,d]
2 6 [,1,4,d,g]
3 a [w,1,NaN,x,y,2]

I need to remove the blank in row 2, and the NaN in row 3 to get:

1 1 [a,b,d,d]
2 6 [1,4,d,g]
3 a [w,1,x,y,2]

I think applying some kind of lambda list comprehension?

df.B=df.B.apply(lambda x: x if x not in ['',np.NaN])

but not working...


You need to work on your comprehension skills :)

import numpy as np

df = pd.DataFrame([
        [1, ['a','b','d','d']],
        [6, ['',1,4,'d','g']],
        ['a', ['w',1,np.nan,'x','y',2]]
    ], columns=['A', 'B'])

df.B.apply(lambda l: [x for x in l if x not in ['', np.nan]])

where l is the current list and x are elements of l.