Jun Jang Jun Jang -3 years ago 63
Python Question

In pandas, how to concatenate horizontally and then remove the redundant columns

Say I have two dataframes.

DF1:
col1, col2, col3,

DF2: col2, col4, col5

How do I concatenate the two dataframes horizontally and have the col1, col2, col3, col4, and col5? Right now, I am doing pd.concat([DF1, DF2], axis = 1) but it ends up having two col2's. Assuming all the values inside the two col2 are the same, I want to have only one columns.

Answer Source

Dropping duplicates should work. Because drop_duplicates only works with index, we need to transpose the DF to drop duplicates and transpose it back.

pd.concat([DF1, DF2], axis = 1).T.drop_duplicates().T
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download