Matthew Gunn Matthew Gunn - 4 years ago 91
Python Question

Replace values in a pandas column that satisfy some condition leads to SettingWithCopyWarning

Let

dtrain
be of type
<class 'pandas.core.frame.DataFrame'>


What is the right way to do the following?

target = dtrain.iloc[:,1] > 0
dtrain.ix[target, 1] = 0


I get the warning:


/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/indexing.py:477:
SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead


Note:
Earlier I create
dtrain
using

dtrain = d.loc[(d.yyyy < 2005) & (d.yyyy >= 1995),:]


This might be a n00b question. I'm reasonably new to Python and don't have a good model in my head yet of what triggers a deep copy and what's just a pointer...

Answer Source

I think first can simplify code from:

dtrain = d.loc[(d.yyyy < 2005) & (d.yyyy >= 1995),:]

to:

dtrain = d[(d.yyyy < 2005) & (d.yyyy >= 1995)]

it is called boolean indexing.

and then add copy - see docs.

dtrain = d[(d.yyyy < 2005) & (d.yyyy >= 1995)].copy()
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download