grma0025 grma0025 - 4 months ago 13
Python Question

Custom wrapper for containers implementing __iter__ and __getitem__

I'm trying to write a custom wrapper class for containers. To implement the iterator-prototocol I provide

__iter__
and
__next__
and to access individual items I provide
__getitem__
:

#!/usr/bin/python
# -*- coding: utf-8 -*-
from __future__ import absolute_import, division, print_function, unicode_literals, with_statement
from future_builtins import *
import numpy as np

class MyContainer(object):

def __init__(self, value):
self._position = 0
self.value = value

def __str__(self):
return str(self.value)

def __len__(self):
return len(self.value)

def __getitem__(self, key):
return self.value[key]

def __iter__(self):
return self

def next(self):
if (self._position >= len(self.value)):
raise StopIteration
else:
self._position += 1
return self.value[self._position - 1]


So far, everything works as expected, e. g. when trying things like:

if __name__ == '__main__':
a = MyContainer([1,2,3,4,5])
print(a)
iter(a) is iter(a)
for i in a:
print(i)
print(a[2])


But I run into problems when trying to use
numpy.maximum
:

b= MyContainer([2,3,4,5,6])
np.maximum(a,b)


Raises "
ValueError: cannot copy sequence with size 5 to array axis with dimension 0
".

When commenting out the
__iter__
method, I get back a NumPy array with the correct results (while no longer conforming to the iterator protocol):

print(np.maximum(a,b)) # results in [2 3 4 5 6]


And when commenting out
__getitem__
, I get back an instance of
MyContainer


print(np.maximum(a,b)) # results in [2 3 4 5 6]


But I lose the access to individual items.

Is there any way to achieve all three goals together (Iterator-Protocol,
__getitem__
and
numpy.maximum
working)? Is there anything I'm doing fundamentally wrong?

To note: The actual wrapper class has more functionality but this is the minimal example where I could reproduce the behaviour.

(Python 2.7.12, NumPy 1.11.1)

Answer

Your container is its own iterator, which limits it greatly. You can only iterate each container once, after that, it is considered "empty" as far as the iteration protocol goes.

Try this with your code to see:

c = MyContainer([1,2,3])
l1 = list(c) # the list constructor will call iter on its argument, then consume the iterator
l2 = list(c) # this one will be empty, since the container has no more items to iterate on

When you don't provide an __iter__ method but do implement a __len__ methond and a __getitem__ method that accepts small integer indexes, Python will use __getitem__ to iterate. Such iteration can be done multiple times, since the iterator objects that are created are all distinct from each other.

If you try the above code after taking the __iter__ method out of your class, both lists will be [1, 2, 3], as expected. You could also fix up your own __iter__ method so that it returns independent iterators. For instance, you could return an iterator from your internal sequence:

def __iter__(self):
    return iter(self.value)

Or, as suggested in a comment by Bakuriu:

def __iter__(self):
    return (self[i] for i in range(len(self))

This latter version is essentially what Python will provide for you if you have a __getitem__ method but no __iter__ method.

Comments