Chris Chris - 4 months ago 12
Python Question

How can I evaluate a value in a Python generator at the time I create the generator, not at the time I iterate it?

The following is my code:

import itertools

i = itertools.chain()
for a in [1, 2, 3]:
i = itertools.chain(i, (a for _ in range(2)))

print(list(i))
[3, 3, 3, 3, 3, 3]


Is there a way I can access the value of
a
when creating the generator, rather than when I iterate it in the
print
statement?

I'd like the output to be
[1,1,2,2,3,3]
, ie, the value of
a
when the generator was created.

This is a trivial problem, but in my case I am iterating 1,000,000 rows in the outer loop, then in the inner loop generating 8 rows for each of those million, so I'm keen to keep it a generator.

Nb. The use case is I'm iterating a table in the outer loop, creating sub-objects for each row, passing the primary key to the sub-objects. The numbers are pretty large, so I want to build up the generator, then bulk insert after the loop (using Django's
Model.objects.bulk_create(generator)
). But by the time I call
bulk_create
the primary key is always set to the last row in the outer loop.

gen = itertools.chain()
for id in ParentModel.objects.all().value_list('id', flat=True)):
gen = itertools.chain(gen, (InnerModel(fk=id) for i in range(10000)))
InnerModel.objects.bulk_create(gen)


All the generated InnerModels point to the last OuterModel in the list.

Answer

If you don't mind wrapping the tuple into a lambda:

>>> import itertools
>>> i = itertools.chain()
>>> for a in [1, 2, 3]:
>>>     i = itertools.chain(i, (lambda x: (x for _ in range(2)))(a))
>>> print(list(i))
[1, 1, 2, 2, 3, 3]

The idea is to copy the value of a of each iteration. lambda's argument can do that. In each iteration, a local variable x is created and assigned with a.