Charles Morris Charles Morris - 6 days ago 4
Python Question

Apply str.lower to Pandas via list Comprehension

I have a dataframe df of the form:

animal fruit
0 "Dog" "Apple"
1 "Cat" "Banana"
2 "Rat" "Grape"


I want to apply str.lower() to all columns (but not headers).

This Works:

for i in df:
df[i] = df[i].str.lower()


How can I write this as a list comphrension?

I tried:

df[i] = [df[i].str.lower() for i in df]


But this does not work and I get a:

TypeError: list indices must be integers, not instancemethod


What must I change within the list comprehension for this to work?

Secondly, is there a more "Pandas-onicy" way of doing this in general, perhaps using the pandas.apply() function?

Many thanks for your help.

Answer

Output from list comprehension is list of Series. So need concat list:

L = [df[i].str.lower() for i in df]
print (L)
[0    dog
1    cat
2    rat
Name: animal, dtype: object, 0     apple
1    banana
2     grape
Name: fruit, dtype: object]

df1 = pd.concat(L, axis=1)
print (df1)
  animal   fruit
0    dog   apple
1    cat  banana
2    rat   grape

Solution with apply:

print (df.apply(lambda x: x.str.lower()))
  animal   fruit
0    dog   apple
1    cat  banana
2    rat   grape

Timings:

df = pd.concat([df]*1000).reset_index(drop=True)
df = pd.concat([df]*1000, axis=1)
df.columns = range(len(df.columns))
#[3000 rows x 2000 columns]
print (df)

In [89]: %timeit (pd.concat([df[i].str.lower() for i in df], axis=1))
1 loop, best of 3: 2.3 s per loop

In [90]: %timeit (df.apply(lambda x: x.str.lower()))
1 loop, best of 3: 2.63 s per loop

In [91]: %timeit (df.stack().str.lower().unstack())
1 loop, best of 3: 5.04 s per loop