Ofek Ron Ofek Ron - 3 years ago 161
Python Question

How to work with zip on 2 generators s.t each returns the same item that changes its state overtime

from collections import deque

def window(seq, n=2):
it = iter(seq)
win = deque((next(it, None) for _ in xrange(n)), maxlen=n)
yield win
append = win.append
for e in it:
append(e)
yield win

a=[1,2,3]
b=[2,4,6]


for d in zip(window(a,2),window(b,2)):
print d
raw_input("Press Enter to continue...")


The output :


(deque([2, 3], maxlen=2), deque([4, 6], maxlen=2))
Press Enter to continue...
(deque([2, 3], maxlen=2), deque([4, 6], maxlen=2))
Press Enter to continue...



What im trying to do here is to capture all the zipped windows of length 2 of
a
and
b
, In other words im expecting the output to be like :


(deque([1, 2], maxlen=2), deque([2, 4], maxlen=2))
Press Enter to continue...
(deque([2, 3], maxlen=2), deque([4, 6], maxlen=2))
Press Enter to continue...



Clearly,
window
is yielding the same object over and over again only with a change in the items it holds, I guess what happens here is that
zip
is creating the iterable before the iteration starts so Im getting the same 2 deque objects zipped twice but the problem is that i get them in their final state.

How would you suggest to fix that without having to yield a copy in the window function?

For example changing window to return a copy would work:

def window(seq, n=2):
it = iter(seq)
win = deque((next(it, None) for _ in xrange(n)), maxlen=n)
yield list(win)
append = win.append
for e in it:
append(e)
yield list(win)


But is less efficient, in my use case a and b are huge and ziping all those copies before the iteration is inefficient...

Answer Source

To avoid zipping the two iterables before beginning the loop, use itertools.izip instead of zip in Python2:

import itertools as IT
import collections 

def window(seq, n=2):
    it = iter(seq)
    win = collections.deque((next(it, None) for _ in range(n)), maxlen=n)
    yield win
    append = win.append
    for e in it:
        append(e)
        yield win

a=[1,2,3]
b=[2,4,6]

for d in IT.izip(window(a,2),window(b,2)):
    print(d)

yields

[deque([1, 2], maxlen=2), deque([2, 4], maxlen=2)]
[deque([2, 3], maxlen=2), deque([4, 6], maxlen=2)]

In Python3, zip returns an iterator (identical to itertools.izip in Python2). This is why the code you posted already works in Python3 without changes.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download