tumbleweed tumbleweed - 14 days ago 6
Python Question

Can't concatenate pandas dataframes with the same length?

This is weird. From the documentation I all ready read how to do concat and merge operations with pandas. Also I all ready know that concatenating to the right side can be done as follows:

df = pd.concat([df1, df2], axis=1)


The issue is that I generated the following dataframes:

In:

links = pd.DataFrame(links, columns=['link'])


Out:

,link
0,http://www.website.io/det/Acetohydroxamic%20Acid%20(Lithostat)%20capsules&st=c&tab=tabs-1
1,http://www.website.io/det/Ammonium%20Chloride%20Injection&st=c&tab=tabs-1
2,http://www.website.io/det/Anagrelide%20Hydrochloride%20Capsules&st=c&tab=tabs-1
3,http://www.website.io/det/Asparaginase%20Erwinia%20Chrysanthemi%20(Erwinaze)&st=c&tab=tabs-1


In:

req_list = req_list.values.tolist()
req_list = [item for sublist in req_list for item in sublist]
req_list = '|'.join(req_list).lower()
intersection['Col3'] = intersection.Col1.str.lower().str.contains(req_list)


Out:

,Col1,Col2,Col3
1,Acetohydroxamic Acid (Lithostat) capsules,New in the Store,False
2,Ammonium Chloride Injection,New in the Store,False
3,Anagrelide Hydrochloride Capsules,New in the Store,True
4,Asparaginase Erwinia Chrysanthemi (Erwinaze),New in the Store,False


So, I just want to concatenate the
link
dataframe column to
intersection
dataframe (note that
link
and
intersection
have 78 instances of length). Thus:

In:

full_table = pd.concat([intersection, links], axis=1)


Out:

Col1,Col2,Col3,Col4
NaN,NaN,NaN,http://www.website.io/det/dsp_ActiveIngredientDetails.cfm?AI=Acetohydroxamic%20Acid%20(Lithostat)%20Tablets&st=c&tab=tabs-1
Acetohydroxamic Acid (Lithostat) Tablets,New in the Store,False,http://www.website.io/det/dsp_ActiveIngredientDetails.cfm?AI=Ammonium%20Chloride%20Injection&st=c&tab=tabs-1
Ammonium Chloride Injection,New in the Store,False,http://www.website.io/det/dsp_ActiveIngredientDetails.cfm?AI=Anagrelide%20Hydrochloride%20Capsules&st=c&tab=tabs-1
Anagrelide Hydrochloride Capsules,New in the Store,True,http://www.website.io/det/dsp_ActiveIngredientDetails.cfm?AI=Asparaginase%20Erwinia%20Chrysanthemi%20(Erwinaze)&st=c&tab=tabs-1


The problem is that as you can see in the above dataframe, it added some
NaN
values. Therefore, which is the correct way of concatenating
links
and
intersection
dataframes?.

Answer

Maybe your indexes don't match up. Try using the ignore_index parameter:

full_table = pd.concat([intersection, links], axis=1, ignore_index=True)