kaminsknator kaminsknator - 6 days ago 5
Python Question

convert pandas dataframe column from hex string to int

I have a very large dataframe that I would like to avoid iterating through every single row and want to convert the entire column from hex string to int. It doesn't process the string correctly with astype but has no problems with a single entry. Is there a way to tell astype the datatype is base 16?

IN:
import pandas as pd
df = pd.DataFrame(['1C8','0C3'], columns=['Command0'])
df['Command0'].astype(int)
OUT:
ValueError: invalid literal for int() with base10: '1C8'


This works but want to avoid row iteration.

for index, row in df.iterrows():
print(row['Command0'])


I'm reading this in from a CSV
pd.read_csv(open_csv, nrows=20)
so if there is a way to read it in and explicitly tell it what the format is then that would be even better!

Answer

You could use apply.

df.Command0.apply(lambda x: int(x, 16))
>>>
0    456
1    195
Name: Command0, dtype: int64

And you can do this with pd.read_csv call using the converters parameter:

df = pd.read_csv("path.txt", converters={"Command0": lambda x: int(x, 16)})
Comments