Generators

You already know how to write a function that builds a list and hands it back with return. Suppose your school's ERP system asks you to write a function that returns every even number, starting from zero, with no upper limit — because the teacher hasn't decided yet how many she'll need. Try to write that with a list:

def even_numbers():
    result = []
    n = 0
    while True:
        result.append(n)
        n += 2
    return result   # this line never runs!

This function never finishes. The while True loop keeps stuffing numbers into result forever, and Python has to keep every single one of them in memory at the same time, because a list is a container that holds all of its items simultaneously. Long before return is ever reached, your program will crash with a memory error. There is no way to build an infinite list — a list, by definition, has a finished size.

And yet "give me the even numbers, one at a time, for as long as I keep asking" is a perfectly reasonable thing to want. A vending machine doesn't manufacture every soft drink it will ever sell and store them all inside itself before switching on — it produces one can the moment you press a button, and worries about the next one only when you press the button again. That is exactly the idea behind a generator: a special kind of function in Python that produces values one at a time, on demand, instead of computing all of them up front and handing back a finished list.

Meet yield: A Function That Pauses Instead of Finishing

The mechanism that makes this possible is one new keyword: yield. Compare these two functions that both produce the first n square numbers:

# Version 1: an ordinary function
def get_squares(n):
    result = []
    for i in range(1, n + 1):
        result.append(i * i)
    return result

squares = get_squares(5)
print(squares)          # [1, 4, 9, 16, 25]

# Version 2: a generator function
def get_squares_gen(n):
    for i in range(1, n + 1):
        yield i * i

squares_gen = get_squares_gen(5)
print(squares_gen)

The second function looks almost identical, but that single swap — yield instead of result.append(...) plus return — changes everything about how Python treats it. The moment a function body contains the word yield anywhere, Python marks the whole function as a generator function. Calling it does not run any of its code. It only creates and hands back a generator object — a paused, dormant version of the function, sitting at the very first line, waiting to be asked for a value. That's why print(squares_gen) does not print [1, 4, 9, 16, 25]. It prints something like:

<generator object get_squares_gen at 0x7f2a4c1d3eb0>

(The exact hexadecimal number after at is just a memory address and will be different every time you run it — ignore it. What matters is that Python is telling you this is a generator object, not a list.)

Tracing a Generator Step by Step

To actually pull values out of a generator, you call the built-in function next() on it. Each call to next() resumes the paused function from exactly where it left off, runs it until the next yield statement, hands back that value, and pauses again — remembering every local variable exactly as it was.

squares_gen = get_squares_gen(5)

print(next(squares_gen))   # 1
print(next(squares_gen))   # 4
print(next(squares_gen))   # 9
print(next(squares_gen))   # 16
print(next(squares_gen))   # 25
print(next(squares_gen))   # StopIteration error!

Let's trace exactly what happens, call by call. The loop variable is i, running from 1 to 5.

That last row is important and easy to miss: a generator does not signal "I'm done" by yielding some special value like None or -1. It signals "I'm done" by raising an exception called StopIteration the moment its function body finishes running without hitting another yield. If you called next() a sixth time on squares_gen above, your program would crash with StopIteration unless you handle it.

How for Loops Use Generators Automatically

Calling next() five times by hand is tedious, and you'll almost never do it in real code. This is exactly what a for loop already does behind the scenes for you — for any list, string, or generator. When you write for value in squares_gen:, Python repeatedly calls next() on squares_gen, and the moment it catches a StopIteration, it quietly ends the loop instead of crashing:

for value in get_squares_gen(5):
    print(value)

Output:

1
4
9
16
25

No StopIteration ever appears on your screen, because the for loop catches it for you internally. This is the normal, idiomatic way to consume a generator — reach for manual next() calls only when you specifically want to control the pacing yourself.

Worked Example: The Fibonacci Generator

Squares are a good first example because each value doesn't depend on the previous one. A more interesting case is the Fibonacci sequence, where each number is the sum of the two before it (0, 1, 1, 2, 3, 5, 8, ...). This is a favourite example precisely because a generator has to remember state — the last two numbers — across pauses:

def fibonacci(n):
    a, b = 0, 1
    for _ in range(n):
        yield a
        a, b = b, a + b

print(list(fibonacci(7)))   # [0, 1, 1, 2, 3, 5, 8]

Let's trace it fully. a and b start at 0 and 1. On every iteration, the function yields the current value of a, then updates both variables in one line — b's old value becomes the new a, and the sum a + b becomes the new b.

The sequence of yielded values reading down the second column is 0, 1, 1, 2, 3, 5, 8 — exactly the first seven Fibonacci numbers, and it matches Python's actual output. Notice what a regular function would have to do instead: build a list of all seven numbers before returning anything. The generator, by contrast, only ever keeps two integers — a and b — in memory at any moment, no matter how far along the sequence it is.

Generator Expressions: A One-Line Shortcut

You already know list comprehensions from earlier chapters, such as [x * x for x in range(1, 6)], which immediately builds the list [1, 4, 9, 16, 25]. Swap the square brackets for round brackets, and you get a generator expression — the compact, one-line way to write a simple generator without a full def block:

squares_list = [x * x for x in range(1, 6)]   # a list, built immediately
squares_gen  = (x * x for x in range(1, 6))   # a generator, built lazily

print(squares_list)   # [1, 4, 9, 16, 25]
print(squares_gen)    # <generator object <genexpr> at 0x...>
print(list(squares_gen))   # [1, 4, 9, 16, 25] — forces it to produce every value

Only the brackets differ, but the behaviour is exactly as different as it was between get_squares and get_squares_gen earlier: one builds everything now, the other builds nothing until asked.

Why Generators Save Memory

Here is the practical payoff. Suppose your school's attendance system logs one line per student per day, and over an academic year that file grows to hundreds of thousands of lines. If you write a function that reads the whole file and returns a list of every line, Python must hold every single line in memory at once before your program can do anything with even the first one:

def read_attendance_as_list(filename):
    lines = []
    with open(filename) as f:
        for line in f:
            lines.append(line.strip())
    return lines

A generator version processes the file one line at a time, and never holds more than the current line in memory, regardless of whether the file has 50 lines or 5 lakh lines:

def read_attendance(filename):
    with open(filename) as f:
        for line in f:
            yield line.strip()

for record in read_attendance("class8_attendance.csv"):
    if "Absent" in record:
        print(record)

The second version starts printing absent students immediately — even before the file has finished being read — because it never waits to build a complete list first. This is the core reason generators exist in real software: a list's memory use grows with how much data it holds, while a generator's memory use stays essentially flat no matter how many values it eventually produces, because it only ever needs to remember where it is, not everywhere it has been.

Misconception 1: "A Generator Is Just a List Written Differently"

It is tempting to think a generator and a list are just two spellings of the same idea, since both can be looped over with for. They are not interchangeable, and the clearest proof is that a generator can only be walked through once. Once it has produced its last value, it is empty — permanently — even though a list holding the same numbers can be re-read as many times as you like:

gen = get_squares_gen(3)
print(list(gen))   # [1, 4, 9]
print(list(gen))   # []  — already exhausted, nothing left!

nums = [1, 4, 9]
print(nums)   # [1, 4, 9]
print(nums)   # [1, 4, 9]  — a list can be read again and again

A generator also cannot be indexed (gen[0] fails, unlike nums[0]), sliced, checked for length with len(), or searched with in without consuming it. It supports exactly one operation well: producing the next value. That narrowness is the price you pay for its constant memory use, and it's a trade-off, not a limitation to work around — you choose a generator specifically when you only need to pass through the data once, in order, such as processing each attendance line or printing each Fibonacci number as it's computed.

Misconception 2: "yield Ends the Function, Just Like return"

The keywords look like they belong to the same family, but they behave completely differently. return permanently exits a function and discards all of its local variables. yield only pauses the function — every local variable, and the exact line it paused on, stays alive in memory, ready to pick up the instant next() is called again. A function can yield hundreds of times across its lifetime; it can only return once, and that return is always its last act.

The diagram below makes this contrast concrete for the squares example: the ordinary function builds and stores every result before handing anything back, while the generator produces exactly one value, hands it out, and forgets everything except its resume point until asked again.

Infinite Sequences: What Only a Generator Can Do

Return to the problem from the very start of this chapter: producing even numbers forever. With yield, the function that crashed earlier becomes perfectly safe:

def even_numbers():
    n = 0
    while True:
        yield n
        n += 2

evens = even_numbers()
print(next(evens))   # 0
print(next(evens))   # 2
print(next(evens))   # 4
print(next(evens))   # 6

This function genuinely never finishes — the while True loop never exits, so this generator could, in principle, be asked for a trillion more values and it would keep producing them. That's fine, because unlike the broken list version, it never tries to store more than the single number it just yielded. This is something a list can never do, no matter how you write it: a list must have a fixed, finite size the moment it exists, while a generator can represent a sequence with no end because it only ever commits to producing the next value when asked, never all of them at once. If you need only the first 10 of these values in real code, you stop calling next() after 10 — the generator simply stays paused forever after that, using no more memory than it did after the first call.

CBSE Exam Pointers

• A generator function is any function whose body contains at least one yield statement. Calling it does not run the body — it returns a generator object.
• yield pauses execution and hands back a value while preserving all local state; return ends execution permanently.
• Values are pulled out with next(generator), or automatically and safely inside a for loop.
• When a generator's code finishes running with nothing left to yield, the next call to next() raises StopIteration. A for loop catches this automatically; manual next() calls do not.
• A generator expression is the compact form: (expression for item in iterable), built with round brackets instead of a list comprehension's square brackets.
• A generator can be iterated only once; after it is exhausted, it stays empty. Lists can be re-iterated any number of times.
• The key advantage of a generator over a list-building function is memory: a generator's memory footprint stays roughly constant regardless of how many values it eventually produces, because it only ever holds its current position and local variables — not a growing collection of past results.

Test Yourself

1. Trace this code and write down exactly what gets printed, in order:

def mystery(n):
    total = 0
    for i in range(1, n + 1):
        total += i
        yield total

for value in mystery(4):
    print(value)

Answer: total accumulates a running sum, and each running sum is yielded as soon as it's updated. With i going 1, 2, 3, 4: after i=1, total=1 (yields 1); after i=2, total=3 (yields 3); after i=3, total=6 (yields 6); after i=4, total=10 (yields 10). Printed output: 1, then 3, then 6, then 10.

2. Rewrite this list-building function as a generator function using yield, without changing what values it eventually produces:

def cubes(n):
    result = []
    for i in range(1, n + 1):
        result.append(i ** 3)
    return result

Answer:

def cubes_gen(n):
    for i in range(1, n + 1):
        yield i ** 3

3. What does this code print, and why does the second line differ from what a beginner might expect?

g = (x for x in range(3))
print(sum(g))
print(sum(g))

Answer: it prints 3 then 0. The generator expression yields 0, 1, 2, and sum(g) consumes all of them the first time, adding up to 3. By the second call, g is already exhausted — it has no values left to give — so sum of an empty generator is 0, not another 3.

4. True or false, with a one-line reason: "You can find the 3rd element of a generator using gen[2], just like a list."

Answer: False. Generators don't support indexing because they don't store their values anywhere — the only way to reach the 3rd value is to call next() on the generator three times (or loop through it), consuming the first two along the way.

5. A function that reads a 2-lakh-line log file needs to find just the first line containing the word "ERROR" and stop. Should it be written to return a list of all lines, or as a generator? Justify your answer in one or two sentences.

Answer: As a generator. Since the search can stop at the very first match, a generator lets the program check each line as it's produced and break out immediately, without ever reading — let alone storing — the remaining lines. A list-returning version would force Python to read and store all 2 lakh lines in memory before the search could even begin, which is wasted work and wasted memory if the match happens to be near the top of the file.

Summary

• A generator function contains yield and produces a generator object when called — no code inside it runs until you ask for a value.
• yield pauses a function and remembers its state; return ends it and forgets everything.
• next() resumes a generator until the next yield; when the function body finishes, the next next() call raises StopIteration, which for loops handle automatically.
• Generator expressions, written with round brackets — (expr for item in iterable) — are the compact one-line equivalent of a full generator function.
• Generators use roughly constant memory no matter how many values they eventually produce, because they hold only their current position, not a growing list of results — this is what makes them suitable for huge files and even infinite sequences.
• The cost of that efficiency is that a generator can be walked through exactly once; once exhausted, it produces nothing more, and it supports none of a list's extras like indexing, slicing, or repeated iteration.

Think About It

Think about this: How would you explain generators to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.

Practice Exercises

Now it is time to practice! Complete these challenges to solidify your understanding:

• Exercise 1: Write a short program that demonstrates the core concept from this chapter. Test it with at least 3 different inputs.
• Exercise 2: Find a real-world example where generators is used in an Indian company (like TCS, Infosys, Flipkart, or ISRO). Write a paragraph explaining the connection.
• Exercise 3: Create a mind-map connecting generators to at least 3 other topics you have studied.