Tom Dawn Tom Dawn - 7 months ago 34
Python Question

python datetime extract hour minute fast

I have a 878000*1 dataframe, where the 1 column is date in several years. I have the following code to create new columns and store year,month,day,hour,min,week in different new columns:

for i in train.index:
train['Year'][i] = train.Dates[i].year
train['Month'][i] = train.Dates[i].month
train['Day'][i] = train.Dates[i].day
train['Hour'][i] = train.Dates[i].hour
train['Min'][i] = train.Dates[i].minute
train['Week'][i] = train.Dates[i].isocalendar()[1]


However this is really slow, my computer has been working overnight for this simple command and it still not finished. I am wondering if there is some faster way that I can use to extract these information without waiting for so long?

Answer

first of all you don't want to do it in loop, you should work with vectorized data:

train['Year'] = train.Dates.dt.year
train['Month'] = train.Dates.dt.month

...