Testing Types

A Report Card Generator That "Worked" — Almost

Imagine your friend Aditi writes a small Python program for her school's computer club. It takes a student's marks out of 100 and prints a grade. She runs it, types in 95, and it correctly prints A. She tries 10, and it correctly prints Fail. No red error text, no crash. Aditi is happy — the program "works."

Two weeks later, a classmate reports something strange: a student who scored exactly 33 marks — the pass mark — was shown as Fail instead of a passing grade. Aditi is confused, because she never saw an error message. The program never crashed. Python never complained about broken syntax. And yet, for one very specific input, it silently produced the wrong answer.

This is the gap that this chapter is about. A program can run from start to finish, produce no error, and still be wrong. Finding out whether a program is wrong — and exactly which inputs expose that wrongness — is not the same skill as writing the program in the first place. That skill, done deliberately and systematically rather than by accident, is called software testing, and it comes in several distinct types, each catching a different category of mistake.

Testing vs. Debugging: Two Different Jobs

Before going further, it helps to separate two words that beginners often blur together: testing and debugging.

Testing is the activity of running a program on purpose, with carefully chosen inputs, to find out whether it behaves the way it is supposed to — and specifically, to try to catch it behaving wrong. A good tester is almost trying to break the program.

Debugging is what happens after testing finds a problem: reading the code, figuring out why it produced the wrong output, and fixing that specific cause.

Testing finds; debugging fixes. You cannot debug a mistake you never noticed, and this is exactly what tripped up Aditi — she never tested the boundary case of exactly 33 marks, so she had nothing to debug, even though the bug was sitting in her code the whole time.

A common misconception is: "If the program runs without printing an error, it must be correct." This confuses two very different kinds of failure. A syntax error (like a missing colon) or a runtime error (like dividing by zero) stops the program and Python tells you about it immediately — these are easy to notice. But a logical error — using the wrong comparison, the wrong formula, the wrong condition — lets the program run to completion and print a confident, wrong answer. Logical errors are the ones testing is specifically designed to hunt down, because the computer will never flag them on its own. As the computer scientist Edsger Dijkstra put it, "testing shows the presence, not the absence, of bugs" — passing your tests does not prove your program is correct, it only proves it survived the particular inputs you happened to try.

Writing Good Test Cases: Equivalence Classes and Boundary Values

Let's fix Aditi's program and use it to learn how to choose test inputs deliberately rather than randomly. Here is a corrected grading function, using a simplified scheme for this chapter (33 and above passes, 75 and above is a B, 90 and above is an A, and marks must be between 0 and 100):

def grade(marks):
    if marks < 0 or marks > 100:
        return "Invalid"
    elif marks >= 90:
        return "A"
    elif marks >= 75:
        return "B"
    elif marks >= 33:
        return "C"
    else:
        return "Fail"

Instead of testing with random numbers, a systematic tester first splits all possible inputs into groups that should behave the same way — these groups are called equivalence classes. For grade(), the input marks naturally splits into: numbers below 0 (invalid), numbers from 0 to 32 (Fail), 33 to 74 (C), 75 to 89 (B), 90 to 100 (A), and numbers above 100 (invalid). Picking one representative value from each class — say -10, 15, 50, 80, 95, 150 — is a reasonable first pass at testing.

But experience shows that bugs love to hide exactly at the edges between classes, not in the middle of them. That's precisely where Aditi's original bug was. So testers add boundary values: the exact numbers where behaviour is supposed to change, plus the numbers just next to them. For our function, the interesting boundaries are 0, 32/33, 74/75, 89/90, and 100/101.

Here is why the boundary check matters, traced against Aditi's original buggy code, which had one small typo — a strict > instead of >=:

def grade_buggy(marks):
    if marks < 0 or marks > 100:
        return "Invalid"
    elif marks >= 90:
        return "A"
    elif marks >= 75:
        return "B"
    elif marks > 33:        # bug: should be >=
        return "C"
    else:
        return "Fail"

Trace grade_buggy(33) by hand, one line at a time: 33 < 0 is False and 33 > 100 is False, so the first branch is skipped. 33 >= 90 is False, skip. 33 >= 75 is False, skip. 33 > 33 is False — because 33 is not strictly greater than 33 — so this branch is also skipped, and execution falls through to else, returning "Fail". A student with exactly the pass mark is told they failed. If a test suite only checked the "middle of the class" values like 15, 50, 80, 95, this bug would sail through undetected, because none of those values sit on the boundary. Only a test case at marks = 33 exposes it.

A well-built test table for just this one function looks like this:

Running the corrected grade() function against every row of this table and confirming the actual output matches the expected output in each row is what it means to test the function properly. Notice that this table has almost twice as many rows as there are equivalence classes — that's normal. Boundary values are deliberately over-represented in good test suites because that's where mistakes concentrate.

Black-Box Testing and White-Box Testing

The test table above was built by looking only at the specification — "0 to 32 is Fail, 33 to 74 is C" and so on — without ever opening the source code. This approach is called black-box testing: you treat the program as a sealed box, feed it inputs, and check whether the outputs match what the requirements say they should be. You don't need to know or care how the code inside is written.

The opposite approach is white-box testing, where the tester deliberately looks at the code's internal structure — its branches, loops, and conditions — and chooses inputs specifically to exercise each part of that structure at least once. For grade(), a white-box tester looks at the five possible return paths (the four elif branches plus the invalid check) and picks one input to trigger each:

Because every one of the five branches was executed by at least one test, this set is said to achieve full branch coverage for this function. Black-box and white-box are not rival techniques where you pick one — real testers combine both. Black-box testing catches disagreements between what the code does and what it was supposed to do; white-box testing makes sure no line of code is left completely unexercised. Note carefully, though: achieving full branch coverage does not by itself guarantee correctness — it only guarantees every branch ran at least once with some input, not that it ran with every input that matters. Our white-box set above (-5, 95, 80, 50, 10) never happens to touch a single boundary value, so it would have completely missed Aditi's original bug at marks = 33, even with 100% branch coverage. Coverage tells you what code you exercised, not whether the answer it produced was right.

The Testing Pyramid: From One Function to a Whole System

So far, everything has focused on testing a single function in isolation. But real software is built from many functions working together, and a working program is much more than the sum of its individually-correct parts. Testing is therefore organised into levels, often drawn as a pyramid, based on how much of the system each test touches at once.

Unit Testing: Checking One Function Alone

A unit is the smallest piece of code you can meaningfully test by itself — usually a single function or method. Unit testing means testing that unit in isolation, feeding it inputs directly and checking its return value, without involving the rest of the program. Everything done with grade() in the sections above — the boundary table, the branch coverage table — was unit testing. Unit tests are cheap to write, run in a fraction of a second, and when one fails it points almost directly at the exact function responsible, which is why programmers write many of them and run them constantly while coding, not just once at the end.

Integration Testing: Where Correctly-Tested Pieces Still Break

Real programs chain functions together. Suppose Aditi extends her program with a second function that takes an entire class's marks and counts how many students land in each grade:

def class_summary(marks_list):
    summary = {"A": 0, "B": 0, "C": 0, "Fail": 0, "invalid": 0}
    for m in marks_list:
        result = grade(m)
        summary[result] += 1
    return summary

Look closely at the dictionary summary: its key for invalid marks is spelled "invalid", all lowercase. But grade(), correctly, returns the string "Invalid" with a capital I. Both functions are individually fine — grade() passed every unit test in our earlier table, and if you unit-tested class_summary() alone using only valid marks like [95, 33, 75], it would also appear to work, producing {"A": 1, "B": 1, "C": 1, "Fail": 0, "invalid": 0} with no error at all.

The crack only appears when you feed the combined pipeline a class list that includes an invalid mark, for example class_summary([95, 33, 32, -5]). Trace it: m = 95 gives result = "A", so summary["A"] becomes 1. m = 33 gives result = "C" (since 33 >= 33), so summary["C"] becomes 1. m = 32 gives result = "Fail", so summary["Fail"] becomes 1. Then m = -5 gives result = "Invalid", and the line summary[result] += 1 tries to look up the key "Invalid" — but the dictionary only contains the lowercase key "invalid". Python dictionary lookups are case-sensitive, so this line crashes with KeyError: 'Invalid'.

This is precisely what integration testing is for: testing that two or more units, each already verified separately, produce correct results when connected and data flows between them. Bugs at this level are rarely about arithmetic — they're about mismatched assumptions: one function returning "Invalid", the other expecting "invalid"; one function returning a list, the other expecting a single number; one using 0-indexed positions, the other expecting 1-indexed ones. Notice too that choosing integration test data follows the exact same discipline as unit testing: you must include a representative from every equivalence class the connected pieces can produce between them, including the awkward "invalid" case — an integration test suite that only ever passed all-valid marks lists would have missed this crash completely, just as surely as the unit tests missed it.

System and Acceptance Testing: Does the Whole App Actually Work?

Once individual functions are unit-tested and their connections are integration-tested, the next level is system testing: running the complete application, exactly the way a real user would, from one end to the other. If Aditi's grading logic becomes part of a full "Report Card Generator" — one that reads a whole class's marks from a file, computes each student's grade, formats a printable report, and calculates the class average — system testing means running that entire pipeline on realistic data and checking that the final report itself is correct: right grades, right formatting, right totals, nothing crashing partway through for a school with, say, 40 students of mixed marks.

The final level, acceptance testing, asks a different question entirely: not "does the code run correctly?" but "does this actually solve the real problem for the real people who will use it?" This is usually done by the intended users themselves, or people representing them — here, perhaps the class teacher or the school's exam coordinator — checking the finished report card generator against real classroom needs: is the layout usable for actually printing report cards, does it handle the specific marks format the school already uses, is it clear enough for a teacher with no programming background to operate without help? A system can pass every unit, integration, and system test and still fail acceptance testing, if it technically works but doesn't fit how the school actually needs to use it.

Two Misconceptions Worth Correcting Directly

First: "If my code ran without an error message, it's correct." As Aditi's story showed, this confuses the absence of a crash with the presence of correctness. Logical errors are silent by nature — the only way to catch them is to deliberately compare actual output against expected output for well-chosen inputs, which is the whole point of writing a test table rather than just eyeballing a couple of runs.

Second: "If I've unit-tested every function and each one passed, the whole program must work." The class_summary() example directly disproves this: grade() passed every unit test, and class_summary() would also pass a shallow unit test, yet the combination crashes on realistic input. Correctness of the parts does not add up automatically to correctness of the whole — that is exactly why integration, system, and acceptance testing exist as separate, necessary levels above unit testing, rather than being redundant extra work.

Practice: Test It Yourself

1. Which single input value would you add to a test suite to check the boundary between "B" and "A" in the corrected grade() function, and what output should it produce?
2. A tester picks test inputs by reading only the assignment sheet describing what the grading scheme should do, without ever opening grade.py. Is this black-box or white-box testing?
3. Trace grade_buggy(75) by hand, line by line. Does the boundary bug in the elif marks > 33 line affect this particular input? Why or why not?
4. A programmer writes five unit tests for grade() and five unit tests for class_summary(), all of which pass. They conclude the report card generator is bug-free. What kind of testing have they skipped, and what specific bug from this chapter would it have caught?
5. A school principal opens the finished report card app, tries it with last term's actual marks sheet, and finds the printed layout doesn't match the format the school is legally required to use. What type of testing uncovered this, and why wouldn't system testing alone have caught it?
6. Write one white-box test input for grade() that specifically exercises the marks >= 75 branch, distinct from any input already used as an example in this chapter.

Answers:

• 1. marks = 90 should produce "A" (and testing 89, which should give "B", alongside it fully covers that boundary).
• 2. Black-box testing — the tester never looked at the code's internal structure, only the required behaviour.
• 3. No effect: at marks = 75, the function returns from the elif marks >= 75 branch before ever reaching the buggy elif marks > 33 line, so it still correctly outputs "B". The bug only shows up when execution actually reaches that specific line, which happens only for marks from 0 up to 74.
• 4. Integration testing was skipped; it would have caught the KeyError: 'Invalid' caused by the mismatched "invalid"/"Invalid" dictionary key when a marks list containing an invalid value was passed through both functions together.
• 5. Acceptance testing, done by the real user (the principal) against real institutional requirements; system testing only checks that the app runs correctly end-to-end on test data, not that its output satisfies external formatting rules the developer may never have known about.
• 6. Any value from 75 to 89 works, for example marks = 82, which should return "B".