Kartik Kartik - 16 days ago 5
Python Question

Better way to forward fill a DataFrame/array with calculations?

I have a DataFrame (or numpy 2D array, if you please) like so:

0 1 2 3 4 5 6 7 8 9
tstamp
2016-09-19 00:00:00 29.0 NaN NaN NaN NaN NaN NaN NaN NaN NaN
2016-09-19 00:01:00 29.0 NaN NaN NaN NaN NaN NaN NaN NaN NaN
2016-09-19 00:02:00 29.0 NaN NaN NaN NaN NaN NaN NaN NaN NaN
2016-09-19 00:03:00 29.0 NaN NaN NaN NaN NaN NaN NaN NaN NaN
2016-09-19 00:04:00 29.0 NaN NaN NaN NaN NaN NaN NaN NaN NaN


I need to fill all
NaN
with this:

for i in range(1, 10):
df.loc[:, i] = const + df.loc[:, i-1].multiply(const)


Where
const
is some constant. Now, obviously the first thing that comes to mind is to use slices:

df.loc[:, 1:] = const + df.loc[:, :-1].multiply(const)


But note that the value of the next column depends on the computed value of the previous column. Is there a way to do this outside the loop?




Ps., although this is totally unnecessary, but:

df.loc[row_slice, col_slice].values


will return a
numpy NDArray
of the sliced frame.

Answer

setup

df = pd.DataFrame(
    np.ones((5, 10), dtype=int) * 29,
    index=pd.date_range('2016-09-19', periods=5, freq='H'),
    columns=range(10))

df

enter image description here


solution

constant = 2
p = np.power(constant, np.arange(1, df.values.shape[1]))
df.iloc[:, 1:] = p * (1 + df.values[:, [0]]) + np.append(0, p[:-1].cumsum())

df

enter image description here