Stefano Potter Stefano Potter - 1 year ago 113
Python Question

TypeError: reduce() of empty sequence with no initial value when merging

I have dataframes which look like this:

df1

Value Hectares_2006
1 10
5 15


df2

Value Hectares_2007
1 20
5 5


df3

Value Hectares_2008
1 22
5 3


and I want to merge them all together by first putting all of the dataframes in a list and then using:

dfs = reduce(lambda left, right: pd.merge(left, right, on=['Value'], how='outer'), list1
)

but this returns:

File "E:/python codes/temp.py", line 32, in <module>
dfs=reduce(lambda left, right: pd.merge(left, right, on=['VALUE'], how='outer'), list1)

TypeError: reduce() of empty sequence with no initial value


my desired output is:

Value Hectares_2006 Hectares_2007 Hectares_2008
1 10 20 22
5 15 5 3


my full code is this, with
files
pathway the pathway to all my files which become the dataframes:

import pandas as pd, os
from simpldbf import Dbf5

list1=[]
files=r'E:\Documents\2015 Summer RA\CDL_in_buffer'
for f in os.listdir(files):
if '.dbf' in f and '.xml' not in f:
table=Dbf5(os.path.join(files,f))
df=table.to_dataframe()
columns=['VALUE', 'CLASS_NAME','Count']
df=df[columns]
if ('2006' in f) or ('2007' in f) or ('2008' in f) or ('2009' in f):
df['Hectares']=df.Count*0.3136
if ('2010' in f) or ('2011' in f) or ('2012' in f) or ('2013' in f) or ('2014' in f) or ('2015' in f):
df['Hectares']=df.Count*0.09
df.drop(['Count'], axis=1, inplace=True)
df=df[df['CLASS_NAME'] .isin (['Corn'])]
df.rename(columns={'CLASS_NAME': 'Crop_' + f.split('.')[0], 'Hectares': 'Hectares_' + f.split('.')[0] }, inplace=True)
list1.append(df)
dfs=reduce(lambda left, right: pd.merge(left, right, on=['VALUE'], how='outer'), list1)

Answer Source

As mentioned in the comments, you need to un-indent the dfs=... line so that it's outside of the for loop. Otherwise, list1 will be empty on the first iteration of the loop if the first file seen doesn't contain .dbf, which will cause the empty sequence error.