theprowler theprowler - 1 year ago 255
Python Question

Pandas - drop columns

I'm aware that dropping a dataframe's columns should be as easy as:

df.drop(df.columns[1], axis=1)
to drop by index

or
dr.dropna(axis=1, how='any')
to drop based on if it contains
NaN
s.

But neither of those works on my dataframe and I'm not sure if that's because of a format issue or data type issue or a misuse or misunderstanding of these commands.

Here is my dataframe:

fish_frame after append new_column: 0 1 2 3 4 \
2 GBE COD NaN NaN 600 NaN
3 GBW COD NaN 11,189 NaN NaN
4 GOM COD NaN 0 NaN Package Deal - $40,753.69
5 POLLOCK NaN NaN 1,103 NaN
6 WHAKE NaN NaN 12 NaN
7 GBE HADDOCK NaN 10,730 NaN NaN
8 GBW HADDOCK NaN 64,147 NaN NaN
9 GOM HADDOCK NaN 0 NaN NaN
10 REDFISH NaN NaN 0 NaN
11 WITCH FLOUNDER NaN 370 NaN NaN
12 PLAICE NaN NaN 622 NaN
13 GB WINTER FLOUNDER 54,315 NaN NaN NaN
14 GOM WINTER FLOUNDER 653 NaN NaN NaN
15 SNEMA WINTER FLOUNDER 14,601 NaN NaN NaN
16 GB YELLOWTAIL NaN 1,663 NaN NaN
17 SNEMA YELLOWTAIL NaN 1,370 NaN NaN
18 CCGOM YELLOWTAIL 1,812 NaN NaN NaN

6 package_deal_column Package_Price new_column
2 NaN Package Deal - $40,753.69 None 600
3 NaN Package Deal - $40,753.69 None 11,1890
4 None Package Deal - $40,753.69 None 0
5 NaN Package Deal - $40,753.69 None 1,103
6 NaN Package Deal - $40,753.69 None 12
7 NaN Package Deal - $40,753.69 None 10,7300
8 NaN Package Deal - $40,753.69 None 64,1470
9 NaN Package Deal - $40,753.69 None 0
10 NaN Package Deal - $40,753.69 None 0
11 NaN Package Deal - $40,753.69 None 3700
12 NaN Package Deal - $40,753.69 None 622
13 None Package Deal - $40,753.69 None 54,31500
14 None Package Deal - $40,753.69 None 65300
15 None Package Deal - $40,753.69 None 14,60100
16 NaN Package Deal - $40,753.69 None 1,6630
17 NaN Package Deal - $40,753.69 None 1,3700
18 None Package Deal - $40,753.69 None 1,81200


And then I have the following lines of code:

fish_frame.drop(fish_frame.columns[1], axis=1)
fish_frame.drop(fish_frame.columns[2], axis=1)
fish_frame.drop(fish_frame.columns[3], axis=1)
fish_frame.drop(fish_frame.columns[4:5], axis=1)
#del fish_frame[4:5] #doesn't work, "TypeError: slice(4, 5, None) is an invalid key"
del fish_frame['Package_Price']
fish_frame.dropna(axis=1, how='any')


And then I printout the dataframe again and it comes out as:

NEW fish_frame: 0 1 2 3 4 \
2 GBE COD NaN NaN 600 NaN
3 GBW COD NaN 11,189 NaN NaN
4 GOM COD NaN 0 NaN Package Deal - $40,753.69
5 POLLOCK NaN NaN 1,103 NaN
6 WHAKE NaN NaN 12 NaN
7 GBE HADDOCK NaN 10,730 NaN NaN
8 GBW HADDOCK NaN 64,147 NaN NaN
9 GOM HADDOCK NaN 0 NaN NaN
10 REDFISH NaN NaN 0 NaN
11 WITCH FLOUNDER NaN 370 NaN NaN
12 PLAICE NaN NaN 622 NaN
13 GB WINTER FLOUNDER 54,315 NaN NaN NaN
14 GOM WINTER FLOUNDER 653 NaN NaN NaN
15 SNEMA WINTER FLOUNDER 14,601 NaN NaN NaN
16 GB YELLOWTAIL NaN 1,663 NaN NaN
17 SNEMA YELLOWTAIL NaN 1,370 NaN NaN
18 CCGOM YELLOWTAIL 1,812 NaN NaN NaN

6 package_deal_column new_column
2 NaN Package Deal - $40,753.69 600
3 NaN Package Deal - $40,753.69 11,1890
4 None Package Deal - $40,753.69 0
5 NaN Package Deal - $40,753.69 1,103
6 NaN Package Deal - $40,753.69 12
7 NaN Package Deal - $40,753.69 10,7300
8 NaN Package Deal - $40,753.69 64,1470
9 NaN Package Deal - $40,753.69 0
10 NaN Package Deal - $40,753.69 0
11 NaN Package Deal - $40,753.69 3700
12 NaN Package Deal - $40,753.69 622
13 None Package Deal - $40,753.69 54,31500
14 None Package Deal - $40,753.69 65300
15 None Package Deal - $40,753.69 14,60100
16 NaN Package Deal - $40,753.69 1,6630
17 NaN Package Deal - $40,753.69 1,3700
18 None Package Deal - $40,753.69 1,81200


With neither the
NaN
drop working nor the index drop working. Only the specific
drop[column name]
command works but I can't do that for every iteration of this script.

I'm very confused and I hope this isn't a very dumb mistake I'm making.

Also, I myself don't fully understand this information but printing
fish_frame.info()
produces:

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 17 entries, 2 to 18
Data columns (total 8 columns):
0 17 non-null object
1 4 non-null object
2 8 non-null object
3 5 non-null object
4 1 non-null object
6 0 non-null object
package_deal_column 17 non-null object
new_column 17 non-null object
dtypes: object(8)
memory usage: 586.0+ bytes


Any help solving this would be appreciated thanks.

Answer Source

If there is no error which I don't see one from your output, you've simply forgotten to use the inplace parameter:

df.drop(df.columns[1], axis=1, inplace=True)
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download