How AI Learns: Training, Testing, and Accuracy

A Conveyor Belt Full of Tomatoes

Picture a packing shed next to a wholesale mandi during peak tomato season. Trucks arrive every hour, each carrying thousands of tomatoes that must be sorted before boxing: the ripe ones go out today, the unripe ones wait a few more days to redden. Right now, a row of workers does this by eye, picking up each tomato, glancing at it, and tossing it left or right. It works, but it is slow, and after six hours of the same motion, tired eyes start making mistakes. A camera fixed above the conveyor belt could, in principle, look at every tomato as it rolls past and decide "ripe" or "unripe" in a fraction of a second. But a camera only produces numbers — pixel colours, brightness values. Something still has to turn those numbers into a decision. That "something" is a computer program, and how that program arrives at the right decision — and how we prove, with an actual number, that it is any good at the job — is exactly what this chapter is about.

Two Very Different Ways to Decide

Suppose the camera reduces each tomato to one measurement: a redness score from 0 to 10, where 0 is fully green and 10 is the deepest red. The obvious first attempt is a rule a person just writes down: "if redness is 7 or higher, call it Ripe; otherwise call it Unripe." This is a hard-coded rule — a human decided the number 7 by intuition, typed it into the program, and the program never questions it again.

Hard-coded rules break down fast in the real world. Different tomato varieties reach ripeness at different shades of red. Lighting on the belt changes through the day. A bruised tomato can look darker than it really is. No single number that a person guesses will fit all of these cases well, and nobody can write down, as a clean formula, every factor that actually decides ripeness.

The alternative is to stop guessing the rule ourselves and instead show the computer many tomatoes whose correct answer is already known — decided by an experienced sorter who cut a few open or tasted them — and let the computer work out which rule best matches those known answers. The computer is not "understanding" ripeness the way a human does. It is systematically searching for a rule that agrees with as many of the known answers as possible. This search process is called training, and the rule it produces is called a model. This is the central shift this chapter asks you to make: in traditional programming, a human supplies the rule; in machine learning, the computer derives the rule from labelled examples.

Three words will recur constantly from here on, so fix them now:

• Feature — a measurable property of the thing we're deciding about. Here, the feature is the redness score.
• Label — the correct answer for a given example, decided by a trustworthy source (a human expert, a lab test, a past outcome). Here, the label is "Ripe" or "Unripe".
• Dataset — a collection of examples, each with its feature(s) and its label attached. Data that comes with labels already known is called labelled data, and it is what training needs.

Building the Labelled Dataset

Suppose the shed's quality inspector sorts 10 sample tomatoes, measures each one's redness with the camera, and writes down the true label for each:

• Redness 1.2 → Unripe
• Redness 2.4 → Unripe
• Redness 3.1 → Unripe
• Redness 3.5 → Ripe (an early-ripening variety that stays pale)
• Redness 3.8 → Unripe
• Redness 4.5 → Unripe
• Redness 6.3 → Ripe
• Redness 7.1 → Ripe
• Redness 8.0 → Ripe
• Redness 9.2 → Ripe

Notice the tomato at redness 3.5. It is genuinely Ripe, even though its redness score is lower than three tomatoes that are actually Unripe. This is not a typo — it is realistic noise. A handful of real tomatoes ripen while still comparatively pale, just as some students score well despite less obvious "signs" of preparation. A good training process has to cope with this kind of exception rather than pretend it doesn't exist, and — as you'll see soon — it becomes an important test of what "learning" actually means.

Training: Searching for the Rule That Makes the Fewest Mistakes

Because there is only one feature here, the simplest possible model is a threshold rule: pick some number t, and predict Ripe whenever redness ≥ t, otherwise predict Unripe. Training means trying many candidate values of t and keeping whichever one is wrong on the fewest of our 10 known examples.

Define accuracy precisely, because we'll use it constantly:

Accuracy = (Number of correct predictions ÷ Total predictions) × 100

Let's hand-check a few candidate thresholds against the 10 tomatoes above, predicting Ripe when redness ≥ t:

• t = 1.0 — every tomato has redness ≥ 1.0, so everything is predicted Ripe. Only the 5 truly Ripe tomatoes are correct; the 5 Unripe ones are all wrong. Accuracy = 5/10 × 100 = 50%.
• t = 3.0 — now redness 1.2 and 2.4 correctly fall below the line as Unripe, but 3.1 does not (it's still ≥ 3.0, wrongly predicted Ripe), and so are 3.5, 3.8, 4.5. Working through all 10: 7 correct. Accuracy = 70%.
• t = 4.5 — the three lowest Unripe tomatoes and the 3.8 Unripe tomato now fall correctly below the line, but 3.5 (truly Ripe) also falls below it and gets wrongly called Unripe, and 4.5 itself (truly Unripe, since 4.5 ≥ 4.5) gets wrongly called Ripe. 8 correct. Accuracy = 80%.
• t = 5.0 — all five Unripe tomatoes (1.2, 2.4, 3.1, 3.8, 4.5) now correctly fall below 5.0, and all four remaining Ripe tomatoes (6.3, 7.1, 8.0, 9.2) correctly sit at or above it. Only the 3.5 tomato is wrong — it's truly Ripe but its redness is below the line, so it gets called Unripe. 9 correct out of 10. Accuracy = 90%.
• t = 6.5 — now 6.3 (truly Ripe) also falls below the line and is wrongly called Unripe, alongside 3.5. 8 correct. Accuracy = 80%, worse again.

So accuracy rises as t moves from 1.0 up to 5.0, then falls again past 5.0. The threshold t = 5.0 gives the fewest mistakes of any value we can try — a single, unavoidable error on the one genuinely unusual tomato. A real training program does not stop at five guesses; it checks many candidate values systematically and keeps the best one. Let's see that as actual code.

The Code: A Computer Searching for the Best Rule

Here is a program that does exactly what we did by hand above, except it checks 19 threshold values (1.0 up to 10.0, in steps of 0.5) instead of five, and never gets tired or careless:

train_data = [
    (1.2, "Unripe"), (2.4, "Unripe"), (3.1, "Unripe"),
    (3.8, "Unripe"), (4.5, "Unripe"), (3.5, "Ripe"),
    (6.3, "Ripe"), (7.1, "Ripe"), (8.0, "Ripe"), (9.2, "Ripe")
]

def predict(redness, threshold):
    return "Ripe" if redness >= threshold else "Unripe"

def accuracy(data, threshold):
    correct = 0
    for redness, actual_label in data:
        if predict(redness, threshold) == actual_label:
            correct += 1
    return correct / len(data) * 100

best_threshold = None
best_accuracy = 0
threshold = 1.0
while threshold <= 10.0:
    acc = accuracy(train_data, threshold)
    if acc > best_accuracy:
        best_accuracy = acc
        best_threshold = threshold
    threshold += 0.5

print("Best threshold found:", best_threshold)
print("Training accuracy:", best_accuracy, "%")

Trace it: predict takes one tomato's redness and a candidate threshold, and returns "Ripe" only if redness is at least that threshold. accuracy runs predict against every example in a dataset, counts how many match the true label, and converts the count into a percentage. The main loop starts a candidate threshold at 1.0 and walks it up to 10.0 in steps of 0.5 — that's 19 values in total — recording whichever threshold has produced the highest training accuracy so far. Because the update only happens when a new accuracy is strictly greater than the current best, the first threshold to reach the maximum score is the one that survives, even if a later threshold ties it. Running this program produces exactly:

Best threshold found: 5.0
Training accuracy: 90.0 %

This matches our hand calculation exactly: 9 correct out of 10, one unavoidable mistake on the pale-but-ripe outlier. The computer did not "understand" tomatoes — it mechanically tried 19 numbers and kept the one with fewest wrong answers. That is what training a simple model really is: systematic search guided by a labelled dataset, not insight.

Why We Never Test on the Data We Trained On

It's tempting to stop here and declare victory: "90% accuracy, ship it to the conveyor belt." But think about what 90% actually measured. It measured how well the rule redness ≥ 5.0 fits the exact same 10 tomatoes the computer used to pick that very rule. That's a bit like a teacher handing you the previous year's board exam paper to practise from, and then giving you that identical paper again as your real exam. Scoring 90% on it would prove almost nothing about whether you actually understood the chapter — you might just have remembered those specific questions and answers. The only fair test of real understanding is a fresh paper, with different numbers, that you have never seen.

Machine learning models face exactly the same trap. A model's job is to work correctly on new tomatoes tomorrow — ones it has never encountered. Measuring its accuracy only on the tomatoes it was trained on tells us nothing about that. So before training even begins, a portion of the labelled data is set aside and locked away — the algorithm is never shown these examples while it searches for the best threshold. This held-back portion is called the test set, and the accuracy measured on it, after training is completely finished, is the number we actually trust.

Testing: The Real Report Card

Suppose 4 more sample tomatoes were measured by the inspector but deliberately kept aside, untouched, during the entire search above:

• Redness 2.0 → Unripe
• Redness 6.8 → Ripe
• Redness 4.8 → Ripe (another slightly pale-but-ripe case)
• Redness 8.5 → Ripe

Now apply the model that training already settled on — redness ≥ 5.0 → Ripe — to these four, without changing the threshold at all:

• 2.0 is below 5.0 → predicted Unripe. True label is Unripe. Correct.
• 6.8 is at or above 5.0 → predicted Ripe. True label is Ripe. Correct.
• 4.8 is below 5.0 → predicted Unripe. True label is Ripe. Wrong — another pale-but-ripe tomato the fixed threshold cannot catch.
• 8.5 is at or above 5.0 → predicted Ripe. True label is Ripe. Correct.

Three correct out of four: test accuracy = 3/4 × 100 = 75%. Extending the same code with this test set and calling accuracy(test_data, best_threshold) prints exactly Test accuracy: 75.0 %, confirming the hand calculation.

Notice the gap: 90% on training data, only 75% on test data. This gap is not a bug in the program — it is the single most important number in this whole chapter. It tells us how much the model's performance drops when it meets tomatoes it has never seen, which is precisely the situation it will face every single day on the real conveyor belt. A model is judged by its test accuracy, never by its training accuracy alone.

The Full Pipeline, in One Picture

Misconception 1: "A Model That Scores 100% on Training Data Is the Best Model"

Imagine a lazy alternative to the threshold rule: instead of searching for a general pattern, the program simply memorises every training tomato's exact redness value alongside its label — a lookup table. If a brand-new tomato happens to have exactly redness 3.5, the table "predicts" Ripe correctly, because it just copies the stored answer. Test this memoriser on the training set itself and it scores a perfect 100%, since every training example is, by definition, stored in its own table.

That perfect score is meaningless. The lookup table never found a rule like "more redness generally means riper" — it found nothing at all beyond the specific 10 numbers it was shown. Hand it a genuinely new tomato with redness 5.7, a value it has never stored, and it has no sensible way to answer; it might guess randomly or default to whatever it happens to fall back on. This failure — scoring very high on training data while scoring poorly on new data — has a name: overfitting. It happens when a model becomes so tailored to the specific examples it trained on that it stops capturing the general pattern those examples were supposed to teach it. A large gap between training accuracy and test accuracy (say, 100% versus 55%) is the classic warning sign of overfitting. Our threshold model, by contrast, was deliberately kept simple — a single number — which is exactly why it generalised reasonably well (90% down to 75%, not 100% down to 20%). The opposite failure also exists: a rule so simple it fails even on training data (imagine predicting "always Unripe" regardless of redness) is called underfitting. Good training aims for the middle ground — a rule general enough to hold up on new data, not so simple that it ignores the pattern entirely.

Misconception 2: "A Higher Accuracy Percentage Always Means a Better Model"

Suppose the packing shed's usual mix, later in the season, is 90 ripe tomatoes for every 10 unripe ones out of every 100. A model that never even looks at redness and just always outputs "Ripe" would be correct on all 90 truly ripe tomatoes and wrong on all 10 truly unripe ones. Its accuracy: 90/100 × 100 = 90% — a number that sounds excellent, printed by a model that learned absolutely nothing about redness at all.

But the entire point of building this system was to catch the unripe tomatoes before they get boxed and shipped to a customer expecting ripe ones. The always-Ripe model catches zero of them — it fails at the one job that actually mattered, while wearing an impressive-looking accuracy score. This is why a single accuracy percentage can be misleading whenever one label is much more common than the other (called an imbalanced dataset): the number can be high simply because guessing the common label is usually right, not because the model learned anything useful. The same trap applies far beyond tomatoes — a medical screening test that always says "healthy" can score above 95% accuracy in a population where genuine disease is rare, while missing every real case. The lesson to carry forward: always ask not just "what is the accuracy?" but "accuracy at correctly identifying what, compared to what would happen if the model did nothing clever at all?" Later years of study formalise this with tools that separately track how well a model catches the rarer, more important label — but the caution to never trust one bare percentage starts here.

Seeing the Decision Boundary

The diagram below plots every training tomato's redness score on a single line, together with the threshold the search settled on:

Every square left of the dashed line and every circle right of it is a correct prediction. The one circled circle sitting to the left of the line is the single training error we calculated by hand earlier — a tomato whose true label disagrees with what its redness score alone can tell us. No threshold, however cleverly chosen, can fix this: the information needed to always get this tomato right (perhaps its softness, or its smell) simply isn't present in the one feature the model was given. This is an important, separate lesson from overfitting: sometimes a model's mistakes come not from a bad training process but from the features it was given being genuinely insufficient to capture the full picture.

The Vocabulary, Tied Together

If your CBSE Artificial Intelligence coursework has introduced the AI Project Cycle, this chapter has walked through two of its central stages in full working detail. Collecting the 14 measured, labelled tomatoes is Data Acquisition. Searching across candidate thresholds to find the rule with fewest training errors is Modelling. Measuring that rule's performance on the untouched test set is Evaluation. Every machine learning system you will meet from here on — whether it sorts tomatoes, recognises handwriting, or filters spam — follows this same shape: gather labelled examples, split off a portion no training step is allowed to see, search for a rule using only the remaining portion, then report performance using the portion kept aside. The specific rule may become far more complex than a single threshold, but the discipline of separating training from testing never goes away.

Check Your Understanding

• 1. A model gets 96% accuracy on its training set and 54% on its test set. Name the problem this pattern describes, and explain in one sentence why it happens.
• 2. A spam-filtering dataset has 980 normal emails and 20 spam emails out of 1000. A model that labels every email "normal" reports 98% accuracy. Explain why this accuracy is misleading, and state the one thing this model is actually failing to do.
• 3. Using the rule "redness ≥ 6.0 → Ripe" (instead of 5.0), recompute the accuracy on the original 10-tomato training set from this chapter. Show which tomatoes are now misclassified and how many are correct.
• 4. In the code in this chapter, what would happen to best_threshold if the comparison on the line if acc > best_accuracy: were changed to if acc >= best_accuracy:? Would the printed training accuracy change? Would the chosen threshold change?
• 5. Explain, in your own words, why a model must never be evaluated using the same examples it was trained on, using an analogy of your own (not the exam-paper one used in this chapter).

Summary

• A feature is a measurable property (redness score); a label is the known correct answer (Ripe/Unripe); a dataset pairs many features with their labels.
• Training is a systematic search for a rule (a model) that makes the fewest mistakes on labelled examples — it is trial-and-error guided by data, not human insight typed in as a fixed rule.
• Accuracy = (correct predictions ÷ total predictions) × 100. It must always be reported alongside which dataset it was measured on.
• The training set is used to search for the rule; the test set is held back, untouched, until the rule is finalised, then used to measure real-world performance.
• High training accuracy with much lower test accuracy is called overfitting — the model memorised specifics instead of learning a general pattern. A rule too simple to even fit the training data is underfitting.
• A single accuracy number can be misleading when one label is far more common than the other — always compare it against what "always guessing the common label" would achieve.
• Some prediction errors are unavoidable with the features available, no matter how well the model is trained — the underlying data may simply not contain enough information to decide every case correctly.

Practice Exercises

Now it is time to practice! Complete these challenges to solidify your understanding:

• Exercise 1: Write a short program that demonstrates the core concept from this chapter. Test it with at least 3 different inputs.
• Exercise 2: Find a real-world example where how ai learns: training, testing, and accuracy is used in an Indian company (like TCS, Infosys, Flipkart, or ISRO). Write a paragraph explaining the connection.
• Exercise 3: Create a mind-map connecting how ai learns: training, testing, and accuracy to at least 3 other topics you have studied.