piRSquared piRSquared - 1 month ago 17
Python Question

how do I drop NA within pandas.DataFrame.query

consider the

df


df = pd.DataFrame(dict(A=[-1, 0, 1, np.nan, 2, np.nan, 3, 4]))


I can drop NA like this

df.dropna()


enter image description here

but how do I do it within the
query
method

df.query('A is not null')


doesn't work... what does?

DSM DSM
Answer

Idea #1: use notnull, which works despite the query syntax:

In [21]: df.query("A.notnull()")
Out[21]: 
     A
0 -1.0
1  0.0
2  1.0
4  2.0
6  3.0
7  4.0

Idea #2: what's distinctive about NaNs is that they're not equal to themselves:

In [22]: df.query("A == A")
Out[22]: 
     A
0 -1.0
1  0.0
2  1.0
4  2.0
6  3.0
7  4.0
Comments