Generators

Functions that yield values one at a time — infinite sequences, huge files, and lazy pipelines without loading everything into memory.

"A generator produces values one at a time and remembers where it left off. It's an iterator built with one magic word: yield."

— ShurAI

The Problem: Lists Load Everything at Once

Generating a million numbers with a list loads all of them into memory immediately:

python — memory-heavy

# This creates a list with 1,000,000 numbers RIGHT NOW
numbers = [n * 2 for n in range(1_000_000)]
# ~8 MB in memory before you use a single value

A generator produces each value on demand — using almost no memory regardless of how large the sequence is.

yield — The Magic Word

Replace return with yield and a regular function becomes a generator function. Each call to next() runs the function until the next yield, then pauses and saves its state:

python

def count_up(limit):
    n = 1
    while n <= limit:
        yield n     # pause here, send n to the caller
        n += 1      # resume here on next call

# Calling count_up() returns a generator object — nothing runs yet
gen = count_up(3)

print(next(gen))   # 1  — runs to first yield, pauses
print(next(gen))   # 2  — resumes, yields 2, pauses
print(next(gen))   # 3  — resumes, yields 3, pauses
# next call would raise StopIteration

How yield pauses and resumes:

next(gen)

→ Function runs until yield n → returns n → freezes right there

next(gen)

→ Resumes from where it froze → runs to next yield → freezes again

next(gen)

→ Function finishes (no more yield) → raises StopIteration → for loop ends

Using a Generator in a for Loop

Generators work seamlessly with for loops — this is the most common way to use them:

python

def squares(limit):
    for n in range(1, limit + 1):
        yield n ** 2

for sq in squares(5):
    print(sq, end=" ")
# 1 4 9 16 25

# Or convert to a list when you need all values at once
print(list(squares(5)))   # [1, 4, 9, 16, 25]

Generator Expressions — One-Liner Generators

Like a list comprehension but with () instead of []. Creates a generator, not a list:

python

# List comprehension — creates list in memory immediately
squares_list = [n**2 for n in range(1000)]   # ~8KB

# Generator expression — generates values one at a time
squares_gen = (n**2 for n in range(1000))    # ~200 bytes!

# Works with sum, max, min directly — no list needed
print(sum(n**2 for n in range(1000)))     # 332833500
print(max(n**2 for n in range(10)))        # 81

Real Example — Reading a Huge File Line by Line

python

def read_lines(filename, keyword):
    """Yield lines from a file that contain a keyword.
    Only one line is in memory at a time — file can be any size."""
    with open(filename) as f:
        for line in f:
            if keyword.lower() in line.lower():
                yield line.strip()

for line in read_lines("server.log", "ERROR"):
    print(line)
# Streams matching lines without loading the whole file

"Use a generator whenever you're producing a sequence of values but don't need them all at once. Generators are how Python handles big data without breaking a sweat."

— ShurAI

🧠 Quiz — Q1

What makes a function a generator function?

🧠 Quiz — Q2

What does calling a generator function (e.g. gen = count_up(5)) actually do?

🧠 Quiz — Q3

What is the memory advantage of a generator over a list comprehension?

🧠 Quiz — Q4

How do you create a generator expression?