vad vad - 2 months ago 17
Python Question

Lazy iterators (generators) with asyncio

I have a blocking, non-async code like this:

def f():
def inner():
while True:
yield read()
return inner()


With this code the caller can choose when to stop the function to generate data. How to change this to async? This solution doesn't work:

async def f():
async def inner():
while True:
yield await coroutine_read()
return inner()


... because
yield
can't be used in
async def
functions. If i remove the
async
from the
inner()
signature, I can't use
await
anymore.

Answer

As noted above, you can't use yield inside async funcs. If you want to create coroutine-generator you have to do it manually, using __aiter__ and __anext__ magic methods:

import asyncio


# `coroutine_read()` generates some data:
i = 0
async def coroutine_read():
    global i
    i += 1
    await asyncio.sleep(i)
    return i


# `f()` is asynchronous iterator.
# Since we don't raise `StopAsyncIteration` 
# it works "like" `while True`, until we manually break.
class f:
    async def __aiter__(self):
        return self

    async def __anext__(self):
        return await coroutine_read()


# Use f() as asynchronous iterator with `async for`:
async def main():
    async for i in f():
        print(i)
        if i >= 3:
            break


if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

Output:

1
2
3
[Finished in 6.2s]

You may also like to see other post, where StopAsyncIteration uses.

Upd:

Starting with Python 3.6 we have asynchronous generators and able to use yield directly inside coroutines.

Comments