Eric Hansen - 1 year ago 140
Python Question

# How to get log rate of change between rows in Pandas DataFrame effectively?

Let's say I have some DataFrame (with about 10000 rows in my case, this is just a minimal example)

``````>>> import pandas as pd

>>> sample_df = pd.DataFrame(
{'col1': list(range(1, 10)), 'col2': list(range(10, 19))})

>>> sample_df

col1  col2
0     1    10
1     2    11
2     3    12
3     4    13
4     5    14
5     6    15
6     7    16
7     8    17
8     9    18
``````

For my purposes, I need to calculate the series represented by
`ln(col_i(n+1) / col_i(n))`
for each
`col_i`
in my DataFrame, where
`n`
represents a row number.
How can I calculate this?

### Background knowledge

I know that I can get the difference between each column in a very simple way using

``````>>> sample_df.diff()

col1  col2
0   NaN   NaN
1     1     1
2     1     1
3     1     1
4     1     1
5     1     1
6     1     1
7     1     1
8     1     1
``````

Or the percentage change, which is
`(col_i(n+1) - col_i(n))/col_i(n+1)`
, using

``````>>> sample_df.pct_change()

col1      col2
0       NaN       NaN
1  1.000000  0.100000
2  0.500000  0.090909
3  0.333333  0.083333
4  0.250000  0.076923
5  0.200000  0.071429
6  0.166667  0.066667
7  0.142857  0.062500
8  0.125000  0.058824
``````

I have just been struggling with a straightforward way to get the direct division of each consecutive column by the previous. Were I to know how to do that even, I could just apply the natural logarithm to every element in the series after the fact.

Currently to solve my problem, I'm resorting to creating another column shifted with row elements down by 1 for each column and then applying the formula between the two columns. It seems messy and sub-optimal to me, though.

Any help would be greatly appreciated!

``````np.log(df.col1 / df.col1.shift())
``````np.log(df / df.shift())