Python Concurrency Basics: Threads, Processes or asyncio?

Before choosing threads, processes or asyncio, answer one question: is your program slow because it is waiting, or because it is computing? Waiting is network calls, disk, databases. Computing is maths, parsing, image processing. The right tool depends almost entirely on that answer, and picking the wrong one can make the program slower, not faster.

Do you need concurrency at all?

Often, no. If a script runs once a night and takes four minutes, making it take one minute changes nothing for anyone. Concurrency adds a new class of bugs: things happening in an order you didn't expect. Use it when the waiting actually costs something: a user staring at a screen, a job that no longer fits its time window, a server that can't keep up.

Waiting work: threads help

Here, time.sleep stands in for a slow network call. Five calls, half a second each:

import time
from concurrent.futures import ThreadPoolExecutor


def fetch(n):
    time.sleep(0.5)          # pretend this is a network request
    return n * 10


start = time.perf_counter()
results = [fetch(n) for n in range(5)]
print("one by one:", results, f"{time.perf_counter() - start:.1f}s")

start = time.perf_counter()
with ThreadPoolExecutor(max_workers=5) as pool:
    results = list(pool.map(fetch, range(5)))
print("5 threads: ", results, f"{time.perf_counter() - start:.1f}s")

one by one: [0, 10, 20, 30, 40] 2.5s
5 threads:  [0, 10, 20, 30, 40] 0.5s

Same results, a fifth of the time. While one thread is waiting, the others can start their own waits. pool.map also returns results in the same order as the inputs, even if they finish in a different order.

Computing work: threads don't help, processes do

Now a CPU-heavy task, counting primes the slow way, four times:

import time
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor


def count_primes(limit):
    count = 0
    for n in range(2, limit):
        if all(n % d for d in range(2, int(n ** 0.5) + 1)):
            count += 1
    return count


if __name__ == "__main__":
    jobs = [60_000] * 4

    start = time.perf_counter()
    [count_primes(j) for j in jobs]
    one = time.perf_counter() - start

    start = time.perf_counter()
    with ThreadPoolExecutor(max_workers=4) as pool:
        list(pool.map(count_primes, jobs))
    threads = time.perf_counter() - start

    start = time.perf_counter()
    with ProcessPoolExecutor(max_workers=4) as pool:
        results = list(pool.map(count_primes, jobs))
    processes = time.perf_counter() - start

    print("primes below 60,000:", results[0])
    print(f"threads vs one by one:   {threads / one:.1f}x the time")
    print(f"processes vs one by one: {processes / one:.1f}x the time")

primes below 60,000: 6057
threads vs one by one:   1...x the time
processes vs one by one: 0...x the time

When this was tested for the course on an 8-core machine, threads took the same time as doing it one by one (1.0x), and four processes took about a third of the time (0.3x). Your numbers depend on your CPU, so run it yourself. The pattern is what matters: for computing, threads give no speed-up in standard Python, and processes do.

The reason is the GIL, the Global Interpreter Lock. In the standard Python build, only one thread runs Python code at a time. Threads still help with waiting, because a thread that is waiting on the network releases the lock. But for pure Python computing, four threads take turns on one lock. Processes each have their own Python and their own lock, so they really run in parallel on separate cores.

Python 3.13 added an optional "free-threaded" build without the GIL. It is a separate build you have to install on purpose, and not every library supports it yet. Unless you know you are on it, assume the GIL is there.

The cost of processes

Processes aren't free. Starting one takes time and memory, and every argument and result has to be copied between processes (pickled). For many tiny tasks, that overhead is bigger than the work, and the process version is slower than a plain loop. Processes pay off when each task does a meaningful amount of computing.

Shared data: the race condition

Threads share memory. That is convenient and dangerous:

import threading

stock = {"KB-01": 0}
lock = threading.Lock()


def restock_safely(times):
    for _ in range(times):
        with lock:
            stock["KB-01"] += 1


threads = [threading.Thread(target=restock_safely, args=(100_000,)) for _ in range(4)]
for t in threads:
    t.start()
for t in threads:
    t.join()
print(stock["KB-01"])

400000

stock["KB-01"] += 1 is really three steps: read, add, write. Without the lock, two threads can read the same value and both write back the same result, and one update is lost. Whether you actually see lost updates without the lock depends on the Python version and timing, which is exactly what makes these bugs so bad: the code passes every test and fails under load. With the lock, only one thread changes the value at a time, and the result is always 400,000.

And asyncio?

asyncio is a third option for waiting work. Instead of several threads, there is one thread and one loop, and your functions say explicitly where they wait (await). It handles thousands of simultaneous waits more cheaply than thousands of threads, and switches only at await, which avoids many race conditions. The catch: everything in the chain has to be async, including the libraries you call. The next lesson covers it.

Choosing

• Waiting, a handful to a few hundred tasks, normal (blocking) libraries like requests: ThreadPoolExecutor. Easiest to add to existing code.
• Waiting, many tasks, async libraries available: asyncio.
• Computing, each task takes a meaningful amount of time: ProcessPoolExecutor.
• Computing on arrays of numbers: check first whether a library like NumPy already does it fast. That often beats any amount of parallel Python.

Common mistakes

• Using threads to speed up maths. Under the GIL, it doesn't work.
• Too many workers. 500 threads hitting one API at once can get you rate-limited or banned. Set max_workers to what the other side can handle.
• Ignoring errors from workers. An exception in a worker is only raised when you collect its result. If you never collect results, errors vanish silently.

What happens if you skip this step

You add threads to a slow calculation and it gets no faster. Or you add processes to a slow network job and it gets slower and uses four times the memory. Or it works, and three weeks later a stock count is off by a few units under load. Spending two minutes on "waiting or computing?" prevents all three.

Your script downloads 200 files and then resizes each image. Which part would you give to threads, which to processes, and why not one tool for both?