Say I have two dataframes.
col1, col2, col3,
DF2: col2, col4, col5
How do I concatenate the two dataframes horizontally and have the col1, col2, col3, col4, and col5? Right now, I am doing pd.concat([DF1, DF2], axis = 1) but it ends up having two col2's. Assuming all the values inside the two col2 are the same, I want to have only one columns.
Dropping duplicates should work. Because drop_duplicates only works with index, we need to transpose the DF to drop duplicates and transpose it back.
pd.concat([DF1, DF2], axis = 1).T.drop_duplicates().T