MaiaVictor MaiaVictor - 28 days ago 6
Python Question

How do I remove rows from a dataframe?

I'm trying to remove outliers from a dataset. In order to do that, I'm using:

df = df[df.attr < df.attr.mean() + df.attr.std()*3]


That seems to work as expected, but, when I do something like:

for i in xrange(df.shape[0]):
print df.attr[i]


Then I get a
KeyError
. Seems like Pandas isn't actually returning a new
DataFrame
with rows dropped. How do I actually remove those rows, and get a fully functional
DataFrame
back?

wwl wwl
Answer

First, find the indices which meet the criteria (which in your case is df.attr < df.attr.mean() + df.attr.std()*3).

x = df.loc[:,attr] < df.attr.mean() + df.attr.std()*3

Next, use DataFrame.drop.

df.drop(x[x].index)

See answers such as How to drop a list of rows from Pandas dataframe? for more information