danche danche - 2 months ago 18
Python Question

Merge multiple DataFrames in an efficient manner

I have plenty of DataFrames need to be merged by

axis=0
, it's important to find some fast way to do this op.

So far, I try
merge
and
append
, but these functions all need assignment after merging like
df = df.append(df2)
and will become slower and slower, is there some another method which can merge in place and more efficient?

Answer Source

Assuming your dataframes have the same index, you can use pd.concat:

In [61]: df1
Out[61]: 
   a
0  1

In [62]: df2
Out[62]: 
   b
0  2

Create a list of dataframes:

In [63]: df_list = [df1, df2]

Now, call pd.concat:

In [64]: pd.concat(df_list, axis=1)
Out[64]: 
   a  b
0  1  2