Rachel Rachel - 1 year ago 60
Python Question

Drop all columns where all values are zero

I have a simple question which relates to similar questions here, and here.

I am trying to drop all columns from a pandas dataframe, which have only zeroes (vertically,

axis=1
). Let me give you an example:

df = pd.DataFrame({'a':[0,0,0,0], 'b':[0,-1,0,1]})

a b
0 0 0
1 0 -1
2 0 0
3 0 1


I'd like to drop column
a
since it has only zeroes.

However, I'd like to do it in a nice and vectorized fashion if possible. My data set is huge - so I don't want to loop. Hence I tried

df = df.loc[(df).any(1), (df!=0).any(0)]

b
1 -1
3 1


Which allows me to drop both columns and rows. But if I just try to drop the columns,
loc
seems to fail. Any ideas?

Answer Source

If it's a matter of 0s and not sum, use df.any:

In [291]: df.T[df.any()].T
Out[291]: 
   b
0  0
1 -1
2  0
3  1

Alternatively:

In [296]: df.T[(df != 0).any()].T # or df.loc[:, (df != 0).any()]
Out[296]: 
   b
0  0
1 -1
2  0
3  1
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download