MatthewMartin MatthewMartin - 9 months ago 64
Python Question

Does chaining a list provide any benefit?

As I dimly understand it, itertools.chain.from_iterable is for combining two or more things that return iterables, i.e. things that use the yield keyword and when used as such they are memory efficient.

I have this code I'm trying to decipher. Someone else wrote it.

all_rows = []
with open(__file__, 'rt') as reader:
for row in reader:
magic = list(itertools.chain.from_iterable(all_rows))
for line in magic:

Wouldn't it be better to just use all_rows directly since it is already an in memory data structure?

Answer Source

You are not chaining one list. You are chaining the contents of that list:

>>> import itertools
>>> all_rows = ['lorem ipsum', 'and so on']
>>> list(itertools.chain.from_iterable(all_rows))
['l', 'o', 'r', 'e', 'm', ' ', 'i', 'p', 's', 'u', 'm', 'a', 'n', 'd', ' ', 's', 'o', ' ', 'o', 'n']

You'd have a point if your code was simply using chain(all_rows) instead of chain.from_iterable(all_rows); then you are absolutely right. There is no point in chaining off a single iterable that is already a list, then convert that chain to a list again.

But the itertools.chain.from_iterable() function takes the contents of an iterable and chains those together. It then depends on the elements in the list.

For your example, chaining multiple strings, the advantage is that you don't have to build a larger list first. You'd otherwise have to use list(''.join(all_rows)).