Abhi Abhi - 9 months ago 71
Python Question

creating pandas data frame from multiple files

I am trying to create a pandas

and it works fine for a single file. If I need to build it for multiple files which have the same data structure. So instead of single file name I have a list of file names from which I would like to create the

Not sure what's the way to append to current data frame in pandas or is there a way for pandas to suck a list of files into a


The pandas concat command is your friend here. Lets say you have all you files in a directory, targetdir. You can:

  1. make a list of the files
  2. load them as pandas dataframes
  3. and concatenate them together


import os
import pandas as pd

#list the files
filelist = os.listdir(targetdir) 
#read them into pandas
df_list = [pd.read_table(file) for file in filelist]
#concatenate them together
big_df = pd.concat(df_list)