NirIzr NirIzr - 1 month ago 15
Python Question

What is the inverse function of itertools.izip in python?

I saw this, and this questions and I'd like to have the same effect, only efficiently done with

.

From
itertool.izip
's documentation:


Like zip() except that it returns an iterator instead of a list


I need an iterator because I can't fit all values to memory so instead I'm using a generator and iterating over the values.

More specifically, I have a generator that generates a three values tuple, and instead of iterating it I'd like to feed three lists of values to three functions, each list represents a single position in the tuple.

Out of those three-tuple-values, only one is has big items (memory consumption wise) in it (lets call it
data
) while the other two contain only values that require only little amount of memory to hold, so iterating over the
data
value's "list of values" first should work for me by consuming the
data
values one by one, and caching the small ones.

I can't think of a smart way to generate one "list of values" at a time, because I might decide to remove instances of a three-value-tuple occasionally, depending on the big value of the tuple.

Using the widely suggested
zip
solution, similar to:

>>> zip(*[('a', 1), ('b', 2), ('c', 3), ('d', 4)])
[('a', 'b', 'c', 'd'), (1, 2, 3, 4)]


Results in the "unpacking argument list" part (*[...]) of this to trigger a full iteration over the entire iterator and (I assume) cache all results in memory, which is as I said, an issue for me.

I can build a mask list (True/False for small values to keep), but I'm looking for a cleaner more pythonic way. If all else fails, I'll do that.

Answer

What's wrong with a traditional loop?

>>> def gen():
...     yield 'first', 0, 1
...     yield 'second', 2, 3
...     yield 'third', 4, 5
...
>>> numbers = []
>>> for data, num1, num2 in gen():
...     print data
...     numbers.append((num1, num2))
...
first
second
third
>>> numbers
[(0, 1), (2, 3), (4, 5)]