retirer retirer - 29 days ago 13
Python Question

pd.to_numeric is not working in python

I'm using Jupyter notebook. Default python kernel. 'numpy' and 'pandas' imported.

This column is named 'Period'. The value contains numbers and text like "Period". The data type is 'object'. I would like to delete the rows in which the value of column 'Period' is not a number.
What I tried to do is to first convert the column from object to numeric.

pd.to_numeric(df['Period'],errors='coerce')


It returned a list of numbers and NaNs. The last line lies:

Name: Period, dtype: float64


If I checked again:

df['Period'].dtype


It returns:

dtype('O')


Apparently the converting did not actually work. Also with the following code, it dropped nothing.

df.dropna(subset=['Period'])


What do you think went wrong?

Answer

Pandas operations like to_numeric don't operate "in-place" by default. I recommend that you assign the result to a column in your dataframe.

df['Period_numbers'] = pd.to_numeric(df['Period'], errors='coerce')

Same goes with dropna.

In most cases you can pass inplace=True to the method or function. But I really do recommend assigning the results instead.