mac mac - 1 year ago 88
Python Question

Weird lambda behaviour in loops

I stumbled upon a behaviour in python that I have a hard time understanding. This is the proof-of-concept code:

from functools import partial

if __name__ == '__main__':
sequence = ['foo', 'bar', 'spam']
loop_one = lambda seq: [lambda: el for el in seq]
no_op = lambda x: x
loop_two = lambda seq: [partial(no_op, el) for el in seq]
for func in (loop_one, loop_two):
print [f() for f in func(sequence)]

The output of the above is:

['spam', 'spam', 'spam']
['foo', 'bar', 'spam']

The behaviour of
is surprising to me as I would expect it to behave as
is an immutable value (a string) that changes at each loop, but
seems to store a pointer to the "looping variable"
, like if the loop would recycle the same memory address for each element of the sequence.

The above behaviour is the same with full-blown functions with a for loop in them (so it is not a list-comprehension syntax).

But wait: there is more... and more puzzling!

The following script works like

b = []
for foo in ("foo", "bar"):
b.append(lambda: foo)

print [a() for a in b]

['bar', 'bar']

But watch what happens when one substitute the variable name

b = []
for a in ("foo", "bar"):
b.append(lambda: a)

print [a() for a in b]

[<function <lambda> at 0x25cce60>, <function <lambda> at 0x25cced8>]

Any idea of what is happening here? I suspect there must be some gotcha related to the underlying C implementation of my interpreter, but I haven't anything else (Jthon, PyPy or similar) to test if this behaviour is consistent across different implementations.

Answer Source

The function lambda: el used in loop_one refers to a variable el which is not defined in the local scope. Therefore, Python looks for it next in the enclosing scope of the other lambda:

lambda seq: [lambda: el for el in seq]

in accordance with the so-called LEGB rule.

By the time lambda: el is called, this enclosing lambda has (of course) already been called and the list comprehension has been evaluated. The el used in the list comprehension is a local variable in this enclosing lambda. Its value is the one returned when Python looks for the value of el in lambda: el. That value for el is the same for all the different lambda: el functions in the list comprehension: it is the last value assigned to el in the for el in seq loop. Thus, el is always 'spam', the last value in seq.

You've already found one workaround, to use a closure such as your loop_two. Another way is to define el as a local variable with a default value:

loop_one = lambda seq: [lambda el=el: el for el in seq]
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download