Advanced Testing: pytest, Mocking, Coverage

Suppose you write a function for a school project that manages a digital wallet — the kind of logic that sits behind a UPI app. It adds money, sends money, and refuses to let you send more than you have. You run it once in the terminal, type in a few numbers, see the right answer, and decide it works. Two weeks later you add a new feature — maybe a cashback bonus — and while editing the file you accidentally change a > to a >= in the balance check. Nothing crashes. Nothing looks wrong. But now the wallet will let a user send money they don't have, right up to and including their exact balance plus zero — actually worse, it silently changes which boundary case is allowed. You won't notice until a user hits that exact edge case, maybe months later, maybe after real money has moved.

This is the problem automated testing solves: not "does my code work right now, when I personally try it," but "does my code keep working, automatically, every single time I change anything, without me having to remember every case I need to re-check by hand." In Grade 9 you have already written functions, used if/else, and raised exceptions. This chapter is about a specific, professional way of checking that code: writing separate, automated test code using a tool called pytest, safely testing code that depends on unpredictable outside things using mocking, and measuring how thorough your tests actually are using coverage. These three ideas are used every single day in real software teams, and they are also exactly the kind of "did you actually understand testing, or did you just memorize the word 'testing'" question that shows up in CBSE Computer Science vivas and project evaluations.

From "try it and see" to a real test function

Here is the wallet code we will use throughout this chapter. Save it as wallet.py:

class InsufficientFundsError(Exception):
    pass

def add_money(balance, amount):
    if amount <= 0:
        raise ValueError("Amount must be positive")
    return balance + amount

def send_money(balance, amount):
    if amount <= 0:
        raise ValueError("Amount must be positive")
    if amount > balance:
        raise InsufficientFundsError("Not enough balance")
    return balance - amount

The informal way to check this is to open a Python shell and type add_money(100, 50), look at the answer 150, and trust your eyes. The problem is that this check disappears the moment you close the shell. It cannot be repeated automatically, it cannot be shared with a teammate, and nobody — including future you — can prove it was ever done. A test fixes this by turning "I checked it and it looked right" into a piece of code that checks itself and reports PASSED or FAILED every time it is run.

pytest is a tool that finds and runs test functions automatically. Its rules are simple and worth memorizing precisely, because they are exactly what makes the automation possible: put your tests in a file whose name starts with test_ (for example test_wallet.py), and inside it write functions whose names also start with test_. pytest scans the folder, imports every such file, and calls every such function. Here is a first test file:

from wallet import add_money, send_money, InsufficientFundsError

def test_add_money_increases_balance():
    assert add_money(100, 50) == 150

def test_send_money_decreases_balance():
    assert send_money(100, 30) == 70

The assert statement is the heart of every test. assert add_money(100, 50) == 150 means: run add_money(100, 50), and if the result is not exactly 150, stop and report a failure. Trace it yourself: inside add_money, amount is 50, which is not <= 0, so the function skips the raise line and returns balance + amount, which is 100 + 50 = 150. Since 150 == 150 is True, the assertion passes silently — a test that passes produces no output at all, which is itself something students often find surprising. Silence means success.

Running pytest test_wallet.py -v from the terminal (the -v flag means "verbose," showing one line per test) gives output like this:

collected 2 items

test_wallet.py::test_add_money_increases_balance PASSED   [ 50%]
test_wallet.py::test_send_money_decreases_balance PASSED  [100%]

2 passed in 0.01s

Good test names are not decoration — test_send_money_decreases_balance tells you exactly what broke just from the failing line, without opening the file. Most professional test functions follow a three-part shape called Arrange-Act-Assert: first set up the inputs you need (Arrange), then call the function under test (Act), then check the result (Assert). In our short examples Arrange and Act are combined into one line, but as tests grow this separation keeps them readable.

Testing that an error happens on purpose

A plain assert cannot test something that is supposed to crash, because if send_money(100, 500) raises InsufficientFundsError, the test function itself crashes before it reaches any assert line — and pytest would (correctly) report that as a problem, even though the code behaved exactly as designed. For this, pytest gives you pytest.raises, used as a with block:

import pytest
from wallet import add_money, send_money, InsufficientFundsError

def test_send_money_raises_when_insufficient():
    with pytest.raises(InsufficientFundsError):
        send_money(100, 500)

def test_add_money_rejects_negative_amount():
    with pytest.raises(ValueError):
        add_money(100, -10)

Read with pytest.raises(InsufficientFundsError): as a promise: "the line inside this block must raise exactly this exception — if it does, the test passes; if it doesn't raise anything, or raises a different exception, the test fails." Trace the first one: inside send_money(100, 500), amount is 500, which is not <= 0, so we move on; then 500 > 100 is True, so InsufficientFundsError is raised — exactly what pytest.raises was waiting for, so the test passes.

Reading a failure — and a common misconception about tests

A common misconception is that a "failing test" always means the source code has a bug. It doesn't — it means the test's expectation did not match reality, and the bug could just as easily be in the test itself. Suppose someone writes this by mistake:

def test_add_money_wrong_expectation():
    assert add_money(100, 50) == 200

pytest's output pinpoints the exact mismatch:

    def test_add_money_wrong_expectation():
>       assert add_money(100, 50) == 200
E       assert 150 == 200
E        +  where 150 = add_money(100, 50)

1 failed in 0.01s

add_money is correct — it really does return 150. The test's expected value of 200 was simply wrong. Reading a traceback is a skill: the line starting with E tells you the actual computed values on both sides of the comparison, so before you touch the source file, always ask "is my test's expectation actually correct?"

There is a second, related distinction CBSE students frequently blur: a test can fail (an assert was false) or it can error (some other exception happened before an assertion was even reached, often in setup code). If a fixture itself crashes, pytest reports "1 error," not "1 failed" — the difference tells you whether to look at your assertion logic or at your setup code.

Fixtures: sharing setup without repeating it

As test files grow, many tests need the same starting data. A fixture is a function, marked with @pytest.fixture, that prepares a value and hands it to any test that asks for it by naming it as a parameter:

import pytest
from wallet import add_money

@pytest.fixture
def starting_balance():
    return 500

def test_add_money_with_fixture(starting_balance):
    assert add_money(starting_balance, 100) == 600

pytest sees that test_add_money_with_fixture has a parameter named starting_balance, notices a fixture with that exact name exists, calls it, and passes its return value 500 into the test. So the test becomes assert add_money(500, 100) == 600, which is true: 500 + 100 = 600. Fixtures matter because if ten tests all need "a wallet that starts with ₹500," changing that starting value later means editing one fixture function instead of ten test bodies.

Testing many inputs at once with parametrize

Consider a grading helper used in a school result system:

def classify_marks(marks):
    if marks >= 90:
        return "A1"
    elif marks >= 75:
        return "A2"
    else:
        return "B"

Writing one separate test function per mark value would be repetitive. @pytest.mark.parametrize runs the same test body once for every value pair you supply:

import pytest
from grading import classify_marks

@pytest.mark.parametrize("marks, expected", [
    (95, "A1"),
    (80, "A2"),
    (60, "B"),
    (90, "A1"),   # boundary: exactly 90
    (75, "A2"),   # boundary: exactly 75
])
def test_classify_marks(marks, expected):
    assert classify_marks(marks) == expected

pytest treats this as five independent test cases, reporting "5 passed" if all match. The boundary values 90 and 75 matter specifically: they test the exact number where the comparison operator flips behaviour. Trace marks = 90: 90 >= 90 is True, so it returns "A1" immediately without even checking the elif. If someone had mistakenly typed marks > 90 instead of marks >= 90, every other test case would still pass — only this boundary case would catch the bug. This is exactly why professional testers deliberately test boundaries, not just "typical" values.

Mocking: testing code that depends on the outside world

Now consider a function that, after debiting money, sends the user an SMS confirmation — the way a real UPI app does:

def send_money_with_sms(balance, amount, sms_client):
    new_balance = send_money(balance, amount)
    sms_client.send(f"Rs.{amount} debited. Balance: Rs.{new_balance}")
    return new_balance

If sms_client were a real SMS gateway connection, running this test would actually send a text message every time you ran your test suite — slow, costly, dependent on network availability, and it would spam a real phone number every single time you pressed "run tests." This is the exact problem mocking solves: replace the real, unpredictable dependency with a fake stand-in object that behaves however you tell it to, and that remembers exactly how it was used so you can check afterward.

Python's built-in unittest.mock module provides Mock, an object that accepts any method call and quietly records it:

from unittest.mock import Mock
from wallet import send_money_with_sms

def test_send_money_with_sms_calls_sms_client():
    fake_sms = Mock()

    result = send_money_with_sms(1000, 200, fake_sms)

    assert result == 800
    fake_sms.send.assert_called_once_with(
        "Rs.200 debited. Balance: Rs.800"
    )

Trace it: fake_sms = Mock() creates a fake object. Inside send_money_with_sms(1000, 200, fake_sms), send_money(1000, 200) runs first — 200 is not <= 0, and 200 > 1000 is False, so it returns 1000 - 200 = 800. Then fake_sms.send(...) is called with the string "Rs.200 debited. Balance: Rs.800" — no real network call happens; fake_sms simply notes down that .send was called and with what argument. Finally fake_sms.send.assert_called_once_with(...) checks that this exact call happened exactly once with exactly this text. No SMS gateway was needed anywhere in this test, and the test runs in milliseconds.

A second, very common mocking situation is code that reaches out to an external service directly, rather than through a parameter — for instance, code that checks something like live train-seat availability by calling a web API:

# irctc_checker.py
import requests

def get_seat_status(train_no):
    response = requests.get(
        f"https://api.example.com/status/{train_no}"
    )
    data = response.json()
    return data["status"]

Here there is no sms_client-style parameter to swap out — requests.get is called directly inside the function. For this, unittest.mock provides patch, which temporarily replaces a named object with a Mock for the duration of a test:

from unittest.mock import patch, Mock
from irctc_checker import get_seat_status

@patch("irctc_checker.requests.get")
def test_get_seat_status(mock_get):
    mock_response = Mock()
    mock_response.json.return_value = {"status": "AVAILABLE"}
    mock_get.return_value = mock_response

    result = get_seat_status("12345")

    assert result == "AVAILABLE"
    mock_get.assert_called_once_with(
        "https://api.example.com/status/12345"
    )

Trace it carefully, because the setup happens before the actual test logic runs: mock_response.json.return_value = {"status": "AVAILABLE"} means "whenever .json() is called on this fake response, hand back this dictionary." mock_get.return_value = mock_response means "whenever requests.get(...) is called, hand back this fake response." Now, when get_seat_status("12345") runs, its call to requests.get(...) is secretly redirected to mock_get, which returns mock_response. Calling .json() on that returns {"status": "AVAILABLE"}, and data["status"] is therefore "AVAILABLE" — matching the assertion.

Here is a genuine, easy-to-fall-into misconception about patch: students often assume you should write @patch("requests.get"), patching the function where it was originally defined. This frequently fails to work as intended. The correct rule is to patch the name where it is looked up when the code runs — in this case, inside the irctc_checker module's own namespace, because that module did import requests and then calls requests.get through its own local reference to the requests module. Hence @patch("irctc_checker.requests.get"), not @patch("requests.get"). Getting this wrong is one of the single most common real-world mocking bugs, and it is worth remembering as a rule, not just a syntax detail: patch where the name is used, not where it is defined.

Coverage: measuring how much of your code your tests actually exercise

Passing tests tell you that whatever code paths they touched behave as expected — but they say nothing about code paths they never touched. Coverage measures exactly this: what fraction of your program's executable lines actually ran during the test suite. The tool for this in Python is coverage.py, typically run as:

coverage run -m pytest
coverage report -m

Let's compute this by hand on classify_marks, so the percentage is not a mystery number but something you can verify yourself. The function has six executable statements: the def line, the if line, return "A1", the elif line, return "A2", and return "B" (the bare else: line itself has no code to execute, so tools like coverage.py don't count it separately). Now suppose our test file contains only this single test:

def test_classify_marks_top_grade():
    assert classify_marks(95) == "A1"

Trace what actually executes: the def line ran once when Python imported the file; calling classify_marks(95) runs the if line, finds 95 >= 90 true, and runs return "A1" — then the function exits immediately. The elif line, return "A2", and return "B" never execute at all, because Python never reaches them. That's 3 executed statements out of 6, which coverage.py would report like this:

Name           Stmts   Miss  Cover
-----------------------------------
classify.py        6      3    50%

Now add the two missing boundary and lower-grade tests from the parametrize example earlier — classify_marks(80) == "A2" and classify_marks(60) == "B". Tracing classify_marks(80): the if is false (80 >= 90 is false), so Python checks the elif — true — and runs return "A2". Tracing classify_marks(60): both if and elif are false, so it falls through to return "B". Between all the tests, every one of the six statements has now executed at least once:

Name           Stmts   Miss  Cover
-----------------------------------
classify.py        6      0   100%

The diagram below shows this same improvement visually — the lines coverage.py marks as executed turn green, and the ones it never saw run stay a warning red, exactly matching the two coverage reports above.

The command coverage html generates a browsable report where your actual source file is shown with exactly this green-and-red highlighting, line by line — the diagram above is a hand-drawn version of what that report looks like.

The most important misconception in this entire chapter

Once students see a coverage percentage, a dangerous idea creeps in: "100% coverage means the code has no bugs." This is false, and it is worth stating precisely why. Coverage only proves that a line of code ran during testing — it says absolutely nothing about whether the assertions checked the right thing, or whether the tests fed in the right inputs. Our classify_marks function reached 100% coverage using only the inputs 95, 80, and 60 — but nobody ever tested what happens with marks = -20 or marks = 500, both of which are nonsense for an exam score, yet the function would silently return "B" for -20 as if a student legitimately scored badly, and "A1" for 500 as if that were a valid mark at all. Every line still ran; the report would still proudly say 100%; and the bug would still be sitting there completely invisible to the coverage number. Coverage tells you what your tests touched. It can never tell you what your tests forgot to check for. Treat a high coverage percentage as a minimum bar for "did I test the obvious paths," never as proof of correctness.

Bringing it together: what each tool is actually for

It helps to keep the three ideas cleanly separated, because CBSE exam and viva questions often test exactly this separation. pytest is the runner: it finds test_* functions and reports pass/fail/error for each, using assert for normal checks and pytest.raises for expected exceptions, with fixtures and parametrize to avoid repeating setup and input values. Mocking, using unittest.mock's Mock and patch, exists specifically for the moment your code touches something slow, costly, or unpredictable — a network call, an SMS gateway, a payment API, a file system, the current date and time — letting you test your own logic without depending on that outside thing actually being available or behaving the same way twice. Coverage, using coverage.py, is a measurement tool that answers one narrow question — "which lines actually ran during the whole test suite" — and should always be read as a floor to raise, never as a certificate of correctness.

Practice: active recall

1. Trace by hand: what does coverage report show for Stmts and Cover if classify_marks is only ever called with marks = 50 across the whole test suite? Work out exactly which of the six statements execute.
2. A teammate writes @patch("requests.get") to mock the API call inside irctc_checker.get_seat_status, and the test still hits the real network. Explain precisely why, using the "patch where it's used" rule.
3. Write a pytest.raises test that checks send_money(100, -5) raises ValueError. Trace through send_money line by line to justify why this specific exception is raised rather than InsufficientFundsError.
4. A test suite for a login function reports 100% coverage but has never once tested what happens when the password field is left empty. Explain, in your own words, why the coverage number does not catch this gap.
5. Predict the exact pytest output — including the assertion line — for assert send_money(100, 30) == 60 given the real send_money function defined in this chapter.

Summary

• An automated test is a function, discovered and run by pytest because its file and function names start with test_, that uses assert to check an actual result against an expected one, and reports PASSED or FAILED without needing a human to eyeball anything.
• Use pytest.raises(SomeException) as a with block to test code that is supposed to raise an exception — a bare assert cannot do this, because the crash would happen before the assertion is reached.
• A failing test is not automatically a bug in your source code — it might be a wrong expectation written into the test itself; a test "error" (versus a "failure") means the crash happened outside your assertions, often in a fixture.
• Fixtures (@pytest.fixture) share setup code across tests; @pytest.mark.parametrize runs one test body against many input/expected-output pairs, which is exactly how you deliberately test boundary values.
• Mocking replaces a slow, costly, or unpredictable dependency (a network call, an SMS API, a payment gateway) with a fake object using unittest.mock.Mock, so your tests check your own logic in milliseconds without depending on the outside world. Use patch to swap out a dependency the code reaches for directly — and always patch the name where it is looked up when the code runs, not where it was originally defined.
• Coverage measures which lines of source code actually executed during your test run, reported as Stmts, Miss, and Cover % by coverage.py. It is a measurement of what ran, never a proof that what ran was checked correctly or that the right inputs were tried — 100% coverage can still hide real bugs in unhandled inputs.

Think About It

Think about this: How would you explain advanced testing: pytest, mocking, coverage to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.