Python asyncio: Wait for Many Things Without Waiting Longer

asyncio lets one thread wait for many things at the same time. While one task waits for a response, the others keep going. Used for the right job, a hundred slow requests take about as long as the slowest one. Used for the wrong job, or with one blocking call in the wrong place, it is slower and harder to read than a plain loop.

The basic pieces

import asyncio


async def fetch_price(sku):
    await asyncio.sleep(1)        # stands in for a network call
    return sku, 100


async def main():
    result = await fetch_price("KB-01")
    print(result)


asyncio.run(main())

('KB-01', 100)

• async def defines a coroutine function. Calling it doesn't run it; it creates a coroutine object that has to be awaited.
• await runs it and, while it waits, lets the event loop run other tasks. You can only use await inside async def.
• asyncio.run(main()) starts the event loop, runs main to the end, and closes the loop. Call it once, at the top of your program.

So far this is just a slower way to write a normal function. The point comes with more than one task.

Many waits at once: gather

import asyncio
import time


async def fetch_price(sku, delay):
    await asyncio.sleep(delay)
    return sku, 100


async def main():
    start = time.perf_counter()
    for sku in ["KB-01", "MS-02", "CB-03"]:
        await fetch_price(sku, 1)
    print(f"one after another: {time.perf_counter() - start:.1f}s")

    start = time.perf_counter()
    results = await asyncio.gather(
        fetch_price("KB-01", 1),
        fetch_price("MS-02", 1),
        fetch_price("CB-03", 1),
    )
    print(f"gather: {time.perf_counter() - start:.1f}s", results)


asyncio.run(main())

one after another: 3.0s
gather: 1.0s [('KB-01', 100), ('MS-02', 100), ('CB-03', 100)]

Awaiting one at a time in a loop waits for each in turn: three seconds. gather starts all three, then waits for all of them together: one second. Results come back in the order you passed them in, regardless of which finished first.

The one rule you can't break: never block the loop

import asyncio
import time


async def bad_fetch(sku):
    time.sleep(1)                 # blocking: freezes the whole event loop
    return sku


async def main():
    start = time.perf_counter()
    await asyncio.gather(bad_fetch("KB-01"), bad_fetch("MS-02"), bad_fetch("CB-03"))
    print(f"gather with time.sleep: {time.perf_counter() - start:.1f}s")


asyncio.run(main())

gather with time.sleep: 3.0s

Same gather, back to three seconds. There is only one thread. time.sleep doesn't hand control back to the loop, so nothing else can run while it sleeps. The same goes for requests.get, a normal database driver, reading a big file, or a long calculation. All of them freeze every other task.

Timeouts: don't wait forever

import asyncio


async def slow_supplier():
    await asyncio.sleep(5)
    return "prices"


async def main():
    try:
        async with asyncio.timeout(1):
            await slow_supplier()
    except TimeoutError:
        print("supplier took longer than 1s, using cached prices")


asyncio.run(main())

supplier took longer than 1s, using cached prices

A network call without a timeout can hang for a very long time. asyncio.timeout (Python 3.11 and newer) cancels the waiting task when time is up. On older versions, use asyncio.wait_for(coro, timeout=1).

Limit how many run at once

gather with 5,000 requests starts 5,000 requests at the same moment. The server on the other side may refuse you, or you run out of connections. A Semaphore caps how many are active:

import asyncio

active = 0
highest = 0


async def fetch(n, limit):
    global active, highest
    async with limit:
        active += 1
        highest = max(highest, active)
        await asyncio.sleep(0.1)
        active -= 1
        return n


async def main():
    limit = asyncio.Semaphore(5)
    results = await asyncio.gather(*(fetch(n, limit) for n in range(50)))
    print(len(results), "done, at most", highest, "at the same time")


asyncio.run(main())

50 done, at most 5 at the same time

Fifty tasks, never more than five in flight. Pick the number based on what the other side allows, not what your computer can do.

When one task fails

import asyncio


async def fetch(sku):
    await asyncio.sleep(0.1)
    if sku == "BAD":
        raise ValueError(f"unknown sku {sku}")
    return sku


async def main():
    results = await asyncio.gather(fetch("KB-01"), fetch("BAD"), fetch("MS-02"),
                                   return_exceptions=True)
    for r in results:
        print("failed:" if isinstance(r, Exception) else "ok:", r)


asyncio.run(main())

ok: KB-01
failed: unknown sku BAD
ok: MS-02

By default, the first error is raised out of gather and you lose the results of the others. With return_exceptions=True, errors come back as values in their place, and you decide what to do with each. Python 3.11 also added asyncio.TaskGroup, which cancels the remaining tasks when one fails. Use it when a single failure means the whole batch is useless.

Common mistakes

• Forgetting await. fetch_price("KB-01") without await does nothing and gives a "coroutine was never awaited" warning.
• Blocking calls inside async functions. Covered above. The most common and the most expensive mistake.
• Using asyncio for computing. It is one thread. A CPU-heavy task blocks the loop like time.sleep does. That job belongs to processes.
• Calling asyncio.run inside code that is already async. It raises an error. Inside async code, await instead.

What happens if you skip this step

You copy an async example, add your own code with requests or a normal database call inside, and it runs no faster than before while looking more complicated. Or it works, without timeouts or limits, until the day a supplier's API is slow and every task hangs at once. asyncio rewards following the rules exactly and punishes breaking them quietly.

Your async script calls 300 supplier APIs and one of them never responds. Without a timeout, when does your script finish?