Jeremy Jeremy - 4 months ago 35
Python Question

How can I normalize the data in a range of columns in my pandas dataframe

Suppose I have a pandas data frame surveyData:

I want to normalize the data in each column by performing:

surveyData_norm = (surveyData - surveyData.mean()) / (surveyData.max() - surveyData.min())

This would work fine if my data table only contained the columns I wanted to normalize. However, I have some columns containing string data preceding like:

Name State Gender Age Income Height
Sam CA M 13 10000 70
Bob AZ M 21 25000 55
Tom FL M 30 100000 45

I only want to normalize the Age, Income, and Height columns but my above method does not work becuase of the string data in the name state and gender columns.


You can perform operations on a sub set of rows or columns in pandas in a number of ways. One useful way is indexing:

# Assuming same lines from your example
cols_to_norm = ['Age','Height']
survey_data[cols_to_norm] = survey_data[cols_to_norm].apply(lambda x: (x - x.mean()) / (x.max() - x.min()))

This will apply it to only the columns you desire and assign the result back to those columns. Alternatively you could set them to new, normalized columns and keep the originals if you want.