jonrsharpe jonrsharpe - 5 months ago 18
Python Question

Implementing an asynchronous iterator

Per PEP-492 I am trying to implement an asynchronous iterator, such that I can do e.g.

async for foo in bar:
...


Here is a trivial example, similar to the one in the docs, with a very basic test of instantiation and async iteration:

import pytest

class TestImplementation:
def __aiter__(self):
return self
async def __anext__(self):
raise StopAsyncIteration


@pytest.mark.asyncio # note use of pytest-asyncio marker
async def test_async_for():
async for _ in TestImplementation():
pass


However, when I execute my test suite, I see:

=================================== FAILURES ===================================
________________________________ test_async_for ________________________________

@pytest.mark.asyncio
async def test_async_for():
> async for _ in TestImplementation():
E TypeError: 'async for' received an invalid object from __aiter__: TestImplementation

...: TypeError
===================== 1 failed, ... passed in 2.89 seconds ======================


Why does my
TestImplementation
appear to be invalid? As far as I can tell it meets the protocol:



  1. An object must implement an
    __aiter__
    method ... returning an asynchronous iterator object.

  2. An asynchronous iterator object must implement an
    __anext__
    method ... returning an awaitable.

  3. To stop iteration
    __anext__
    must raise a
    StopAsyncIteration
    exception.




This is failing with the latest released versions of Python (3.5.1),
py.test
(2.9.2) and
pytest-asyncio
(0.4.1).

Answer

If you read a little further down the documentation it mentions that (emphasis mine):

PEP 492 was accepted in CPython 3.5.0 with __aiter__ defined as a method, that was expected to return an awaitable resolving to an asynchronous iterator.

In 3.5.2 (as PEP 492 was accepted on a provisional basis) the __aiter__ protocol was updated to return asynchronous iterators directly.

Therefore for versions prior to 3.5.2 (released 2016/6/27) the documentation is slightly out of step with how to write a working asynchronous iterator. The fixed version for 3.5.0 and 3.5.1 looks like:

class TestImplementation:
    async def __aiter__(self):
  # ^ note
        return self
    async def __anext__(self):
        raise StopAsyncIteration

This was introduced on closing bug #27243 and is a little clearer in the data model documentation, which also suggests a way of writing backwards compatible code.