Python Context Managers: A Deep Dive

What Are Context Managers Really Doing?

You have used with open('file.txt') as f: hundreds of times. But do you know what happens behind the scenes? Context managers implement a protocol defined by two dunder methods: __enter__ and __exit__. When Python encounters a with statement, it calls __enter__ to set up a resource and __exit__ to tear it down, even if an exception occurs inside the block.

with open("data.txt", "r") as f:
    content = f.read()
# f is automatically closed here, even if an exception was raised

This is roughly equivalent to:

f = open("data.txt", "r")
try:
    content = f.read()
finally:
    f.close()

The with statement guarantees cleanup. That is its entire purpose.

The Context Manager Protocol

Any object that implements __enter__ and __exit__ can be used with the with statement. Here is the simplest custom context manager:

class Timer:
    """Measure execution time of a code block."""

    def __enter__(self):
        import time
        self.start = time.perf_counter()
        return self  # This becomes the 'as' variable

    def __exit__(self, exc_type, exc_val, exc_tb):
        import time
        self.elapsed = time.perf_counter() - self.start
        print(f"Elapsed: {self.elapsed:.4f}s")
        return False  # Don't suppress exceptions

with Timer() as t:
    total = sum(range(1_000_000))

print(f"Stored elapsed time: {t.elapsed:.4f}s")

The __exit__ method receives three arguments describing any exception that occurred. If it returns True, the exception is suppressed. Returning False (or None) lets the exception propagate.

Understanding exit Parameters

The __exit__ method always receives exactly three arguments:

class ErrorHandler:
    def __enter__(self):
        return self

    def __exit__(self, exc_type, exc_val, exc_tb):
        if exc_type is ValueError:
            print(f"Caught ValueError: {exc_val}")
            return True  # Suppress the ValueError
        if exc_type is not None:
            print(f"Unexpected error: {exc_type.__name__}: {exc_val}")
        return False  # Let other exceptions propagate

with ErrorHandler():
    raise ValueError("bad input")
# Prints "Caught ValueError: bad input" and continues

with ErrorHandler():
    raise TypeError("wrong type")
# Prints the message, then the TypeError propagates

When no exception occurs, all three parameters are None.

Generator-Based Context Managers with contextlib

Writing a full class for every context manager is verbose. The contextlib module provides @contextmanager, which lets you write context managers as generator functions.

from contextlib import contextmanager
import os

@contextmanager
def working_directory(path):
    """Temporarily change the working directory."""
    original = os.getcwd()
    try:
        os.chdir(path)
        yield path  # This value is bound to 'as'
    finally:
        os.chdir(original)

with working_directory("/tmp") as p:
    print(f"Now in: {os.getcwd()}")
# Back to original directory

Everything before yield is your __enter__ logic. Everything after yield (inside finally) is your __exit__ logic. The yielded value becomes the as target.

Practical Example: Database Transaction Manager

Here is a realistic context manager for managing database transactions:

from contextlib import contextmanager

@contextmanager
def transaction(connection):
    """Manage a database transaction with automatic commit/rollback."""
    cursor = connection.cursor()
    try:
        yield cursor
        connection.commit()
    except Exception:
        connection.rollback()
        raise
    finally:
        cursor.close()

# Usage
# with transaction(db_conn) as cursor:
#     cursor.execute("INSERT INTO users (name) VALUES (?)", ("Alice",))
#     cursor.execute("INSERT INTO logs (action) VALUES (?)", ("user_created",))
# Both inserts commit together, or both roll back on error

Nesting Context Managers with ExitStack

When you need a dynamic number of context managers, contextlib.ExitStack is invaluable.

from contextlib import ExitStack

def process_files(filenames):
    with ExitStack() as stack:
        files = [
            stack.enter_context(open(fname))
            for fname in filenames
        ]
        # All files are open here
        for f in files:
            print(f.readline())
    # All files are closed here, regardless of exceptions

ExitStack also supports registering arbitrary cleanup callbacks:

from contextlib import ExitStack

def setup_resources():
    stack = ExitStack()
    try:
        db = stack.enter_context(connect_db())
        cache = stack.enter_context(connect_cache())
        stack.callback(print, "All resources cleaned up")
        return stack.pop_all()  # Transfer ownership to caller
    except Exception:
        stack.close()  # Clean up on failure
        raise

The pop_all() method transfers all registered cleanups to a new ExitStack, giving the caller responsibility for cleanup.

Reentrant and Reusable Context Managers

Not all context managers can be used multiple times. The distinction matters.

from contextlib import contextmanager

# This is a SINGLE-USE context manager
@contextmanager
def single_use():
    print("Setting up")
    yield
    print("Tearing down")

cm = single_use()
with cm:
    pass  # Works fine
# with cm:  # This would raise RuntimeError!

# For reusable context managers, use a class
class Reusable:
    def __enter__(self):
        print("Enter")
        return self

    def __exit__(self, *exc):
        print("Exit")
        return False

cm = Reusable()
with cm:
    pass  # First use
with cm:
    pass  # Works fine - second use

Suppressing Exceptions with contextlib.suppress

Instead of writing empty except blocks, use suppress:

from contextlib import suppress
import os

# Instead of:
try:
    os.remove("temp.txt")
except FileNotFoundError:
    pass

# Write:
with suppress(FileNotFoundError):
    os.remove("temp.txt")

This is cleaner and communicates intent more clearly.

Async Context Managers

Python 3.10+ makes async context managers straightforward:

from contextlib import asynccontextmanager

@asynccontextmanager
async def async_db_session(url):
    session = await create_session(url)
    try:
        yield session
    finally:
        await session.close()

# Usage:
# async with async_db_session("postgres://...") as session:
#     result = await session.execute(query)

Best Practices

1. Always use context managers for resource cleanup — files, locks, connections, temporary state.
2. Prefer @contextmanager for simple cases; use classes when you need reusability or complex state.
3. Never suppress exceptions silently unless you have a very good reason.
4. Use ExitStack when dealing with a variable number of resources.
5. Return False from __exit__ unless you intentionally want to swallow exceptions.

Wrapping Up

Context managers are one of Python’s most elegant features. They guarantee resource cleanup, make error handling cleaner, and communicate intent clearly. Whether you use classes, generators, or ExitStack, mastering context managers will make your code more robust and Pythonic.