Generators in Python: How They Work

Processing a sequence can mean handling each result as it arrives rather than saving every result in a list. A Python generator produces its next value when code asks for it.

I’ll show how to use a Python generator and when a generator expression fits a short calculation.

TL;DR

A generator produces values as code asks for them, so you can process a sequence without first building a list of every result.

• Put yield in a function to make it a generator function.
• Calling that function returns a one-pass generator object.
• Use a list when you need indexing or repeated access.

What Is a Generator in Python?

A generator is an iterator that produces its next value on demand because a function containing yield returns one value and suspends with its local state saved.

A regular function stops at return. For a closer look at the iterator protocol, see the Python iterator guide.

How to Create and Use a Python Generator

Define a function with yield, then consume its object with a loop or a call to next. This complete example also uses a generator expression for a filtered calculation.

Step 1: Yield values and consume them

The return annotation Iterator[int] tells type-checking tools that the function yields integers. Python does not enforce that annotation at runtime. Save this as generator_walkthrough.py and run the command shown in the terminal screenshot.

from collections.abc import Iterator


def squares(limit: int) -> Iterator[int]:
    for number in range(1, limit + 1):
        yield number * number


values = squares(4)
print("First value:", next(values))
print("Remaining values:", list(values))
print("After exhaustion:", list(values))
print("Fallback after exhaustion:", next(values, "done"))

even_squares = (
    number * number
    for number in range(1, 5)
    if number % 2 == 0
)
print("Even squares:", list(even_squares))

print("Sum of squares:", sum(number * number for number in range(1, 5)))

First value: 1
Remaining values: [4, 9, 16]
After exhaustion: []
Fallback after exhaustion: done
Even squares: [4, 16]
Sum of squares: 30

I ran this file for this refresh, and the output shows the first value followed by the remaining squares. The empty list confirms that the same object is exhausted. The expression yields even squares, while sum consumes its expression without building an intermediate list.

Step 2: Choose an expression for a short transform

The parenthesized expression in the file applies a calculation and a filter as iteration requests values. It suits a short pipeline, while a named function is clearer when the operation needs several statements or its own explanation. The call that builds a list above is for visible output, not a requirement for using an expression.

Common Generator Edge Cases

A generator suits a single-pass stream because the consumer can handle each result as it arrives. Choose a list when the result must support indexing, repeated passes or a stable snapshot.

Calling next manually raises StopIteration after the final value, unless a default is supplied as in the example. A for loop handles normal exhaustion. Generators can lower the amount of retained output data, but they do not guarantee lower total memory or faster execution for every workload.

Review the list comprehension guide for a syntax comparison. The Python documentation on generator annotations covers send and return types.

Conclusion: Produce Values Only When Needed

Set a stopping condition when the source has no natural end. The Python yield reference above describes the function’s suspended state.

• Python yield expressions
• Python Iterator type
• Python generator annotations

FAQ

These answers cover a couple of details that help when a generator becomes part of a function’s interface.