I was reading about generator and iterators and the role of
'__next__' in dir(mygen)
'__next__' in dir(mylist)
'__next__' in dir (mylist.__iter__())
mygen() and __iter__() ?
The special methods
__next__ are part of the iterator protocol to create iterator types. For this purpose, you have to differentiate between two separate things: Iterables and iterators.
Iterables are things that can be iterated, usually, these are some kind of container elements that contain items. Common examples are lists, tuples, or dictionaries.
In order to iterate an iterable, you use an iterator. An iterator is the object that helps you iterate through the container. For example, when iterating a list, the iterator essentially keeps track of which index you are currently at.
To get an iterator, the
__iter__ method is called on the iterable. This is like a factory method that returns a new iterator for this specific iterable. A type having a
__iter__ method defined, turns it into an iterable.
The iterator generally needs a single method,
__next__, which returns the next item for the iteration. In addition, to make the protocol easier to use, every iterator should also be an iterable, returning itself in the
As a quick example, this would be a possible iterator implementation for a list:
class ListIterator: def __init__ (self, lst): self.lst = lst self.idx = 0 def __iter__ (self): return self def __next__ (self): try: item = self.lst[self.idx] except IndexError: raise StopIteration() self.idx += 1 return item
The list implementation could then simply return
ListIterator(self) from the
__iter__ method. Of course, the actual implementation for lists is done in C, so this looks a bit different. But the idea is the same.
Iterators are used invisibly in various places in Python. For example a
for item in lst: print(item)
This is kind of the same to the following:
lst_iterator = iter(lst) # this just calls `lst.__iter__()` while True: try: item = next(lst_iterator) # lst_iterator.__next__() except StopIteration: break else: print(item)
So the for loop requests an iterator from the iterable object, and then calls
__next__ on that iterable until it hits the
StopIteration exception. That this happens under the surface is also the reason why you would want iterators to implement the
__iter__ as well: Otherwise you could never loop over an iterator.
As for generators, what people usually refer to is actually a generator function, i.e. some function definition that has
yield statements. Once you call that generator function, you get back a generator. A generator is esentially just an iterator, albeit a fancy one (since it does more than move through a container). As an iterator, it has a
__next__ method to “generate” the next element, and a
__iter__ method to return itself.