Python Closures: Functions That Remember

A closure is an inner function that remembers the variables of the function that created it, even after that outer function has finished. That one idea is behind decorators, callback factories, and a lot of code that looks like magic until you see it.

Step 1: functions are values

In Python, a function is an object like any other. You can store it in a variable, pass it to another function, and return it from a function:

def shout(text):
    return text.upper() + "!"


say = shout              # no brackets: the function itself, not a call
print(say("hello"))
print(list(map(shout, ["a", "b"])))

HELLO!
['A!', 'B!']

Step 2: a function that makes functions

def make_vat(rate):
    def add_vat(amount):
        return round(amount * (1 + rate), 2)
    return add_vat


ph_vat = make_vat(0.12)
sg_gst = make_vat(0.09)

print(ph_vat(1000))
print(sg_gst(1000))

1120.0
1090.0

make_vat finished running the moment it returned. Its local variable rate should be gone. But ph_vat still knows its rate is 0.12, and sg_gst knows its rate is 0.09. Each inner function kept a reference to the rate it was created with. That is a closure. (The rates are example values for the exercise. Check the current rate before using one in real code.)

You can even see what was captured:

def make_vat(rate):
    def add_vat(amount):
        return round(amount * (1 + rate), 2)
    return add_vat


ph_vat = make_vat(0.12)
print(ph_vat.__code__.co_freevars)
print(ph_vat.__closure__[0].cell_contents)

('rate',)
0.12

You will rarely need these attributes in real code. They are here to show that nothing magic is going on: the function carries its captured variables with it.

Step 3: changing a captured variable needs nonlocal

def make_counter():
    count = 0

    def increment():
        count += 1
        return count

    return increment


counter = make_counter()
counter()

Traceback (most recent call last):
...
UnboundLocalError: cannot access local variable 'count' where it is not associated with a value

Reading a captured variable is fine. Assigning to it is not, by default: count += 1 makes Python treat count as a new local variable of increment, which has no value yet. nonlocal says "use the one from the enclosing function":

def make_counter():
    count = 0

    def increment():
        nonlocal count
        count += 1
        return count

    return increment


a = make_counter()
b = make_counter()
print(a(), a(), a())
print(b())

1 2 3
1

a and b each have their own count. Every call to make_counter creates a new one. That is private state without a class.

The late-binding trap

This one catches experienced people:

handlers = []
for level in ["low", "medium", "high"]:
    handlers.append(lambda: f"alert level: {level}")

print([h() for h in handlers])

['alert level: high', 'alert level: high', 'alert level: high']

A closure captures the variable, not its value at that moment. All three lambdas look up level when they are called, and by then the loop has finished with level = "high". Two common fixes:

from functools import partial

# Fix 1: default argument, evaluated when the lambda is created
handlers = [lambda level=level: f"alert level: {level}" for level in ["low", "medium", "high"]]
print([h() for h in handlers])


# Fix 2: a factory function, so each closure gets its own variable
def make_handler(level):
    return lambda: f"alert level: {level}"


handlers = [make_handler(level) for level in ["low", "medium", "high"]]
print([h() for h in handlers])

# Not a closure at all, but often the cleanest: functools.partial
def alert(level):
    return f"alert level: {level}"


handlers = [partial(alert, level) for level in ["low", "medium", "high"]]
print([h() for h in handlers])

['alert level: low', 'alert level: medium', 'alert level: high']
['alert level: low', 'alert level: medium', 'alert level: high']
['alert level: low', 'alert level: medium', 'alert level: high']

Closure or class?

Anything a closure does, a small class can do too. Use a closure when there is one behaviour and a little state, like make_vat or make_counter. Use a class when there are several methods, when other code needs to read the state, or when the state needs to be reset or saved. A closure with three nonlocal variables is a class pretending not to be one.

Common mistakes

• Calling instead of returning. return add_vat() returns the result of calling it, not the function. Return add_vat without brackets.
• Using global instead of nonlocal. global skips the enclosing function and goes to the module level. Different variable, confusing bugs.
• Capturing a big object by accident. A closure keeps everything it references alive. Capturing a large list or a database connection means it isn't freed while the closure exists.

What happens if you skip this step

Decorators stay a pattern you copy without understanding. Then one doesn't work the way you expect, and you can't tell why, because the reason is a closure capturing a variable in a way you didn't expect. The next lesson builds on exactly this.

Without running it: if you create three counters with make_counter() and call the second one twice, what does the first one return the next time you call it?