The Problem: Sorting Snacks Without Tasting Them
Imagine you run the nutrition-labelling desk for a college canteen app that lists snacks from forty different vendors across the city. Every week, new dishes get uploaded — kachori from one stall, dhokla from another, momos from a third. Your app promises a "Low-Oil" filter for health-conscious students, which means every single dish has to be tagged as either Fried (deep-fried, high oil absorption) or Steamed (steamed, boiled, or lightly pan-cooked, low oil absorption). Tasting every dish yourself, every week, forever, does not scale. You want a program that looks at a few measurable facts about a dish and decides which bucket it belongs to, automatically, the moment a vendor uploads something new.
This is a classification problem: given an object, assign it to one of a fixed set of categories. It is different from a problem like "predict tomorrow's temperature," where the answer is a number that can be anything on a continuous scale — that is called regression. Here, the answer is always one of two labels. A classifier is simply a function: it eats some facts about a dish and spits out a label. Our job for this chapter is to build that function from real examples, check whether it actually works, and then turn it into something that can be plugged into a live app — the full journey "from data to deployment."
Turning Dishes Into Numbers: Features
A computer cannot compare the words "samosa" and "idli" the way your tongue can. It can only do arithmetic. So the first job in any machine learning project — and this is true whether you are classifying snacks or spam emails or X-ray images — is to turn each example into a set of numbers called features. A feature is anything measurable that might help distinguish one category from another.
For our two categories, two features are especially informative:
- Oil content — grams of oil absorbed per 100 g of the dish (roughly 0 for a plain steamed item, up to 20 for something deep-fried in a heavy batter).
- Spice level — a simple 0–6 rating based on how much chilli, pepper, and tempering the dish typically carries.
These numbers are illustrative teaching values, not laboratory measurements — the point is to see how the arithmetic works, not to memorise exact nutrition facts. Here is our starting dataset of six known dishes, each already correctly labelled by a human:
- Samosa — oil 18, spice 4 — Fried
- Pakora — oil 20, spice 5 — Fried
- Vada — oil 16, spice 3 — Fried
- Idli — oil 1, spice 2 — Steamed
- Dhokla — oil 2, spice 3 — Steamed
- Momos (steamed) — oil 1, spice 4 — Steamed
Once every dish has two numbers attached to it, each dish becomes a point that we can plot on a graph — oil content along one axis, spice level along the other. This graph is called feature space. Any new dish, once measured the same way, lands somewhere on that same graph. The entire idea behind our classifier is beautifully simple: dishes of the same type tend to land near each other in feature space, because frying and steaming leave a genuinely different numeric fingerprint. Classification becomes a geometry problem — figure out which cluster of points a new point is closest to.
Measuring "Closeness": Distance Is Pythagoras in Disguise
To say one point is "closer" to another, we need a precise way to measure distance between two points on a graph — and you already know the tool for this from geometry class: the Pythagoras theorem. If you know the horizontal gap and the vertical gap between two points, those two gaps are the two legs of a right triangle, and the straight-line distance between the points is the hypotenuse.
For two points (x₁, y₁) and (x₂, y₂), the horizontal leg has length (x₁ − x₂) and the vertical leg has length (y₁ − y₂). By Pythagoras, the hypotenuse — the straight-line distance — is:
distance = sqrt( (x1 - x2)^2 + (y1 - y2)^2 )
This is called the Euclidean distance, and it is nothing more than the Pythagoras theorem rearranged to solve for the hypotenuse. Let's compute one by hand before writing any code. Take Samosa (18, 4) and Idli (1, 2):
- Horizontal gap: 18 − 1 = 17
- Vertical gap: 4 − 2 = 2
- Sum of squares: 17² + 2² = 289 + 4 = 293
- Distance: √293 ≈ 17.12
Samosa and Idli are numerically far apart — which makes sense, since one is heavily fried and the other is barely oiled at all. Now compare Idli (1, 2) with Dhokla (2, 3): horizontal gap 1, vertical gap 1, sum of squares 1 + 1 = 2, distance √2 ≈ 1.41. Idli and Dhokla sit almost on top of each other in feature space — both are low-oil, mildly spiced, and both are Steamed. This is exactly the pattern a classifier can exploit.
The k-Nearest Neighbours Algorithm
The algorithm we will build is called k-Nearest Neighbours, or k-NN for short. Given a new, unlabelled dish, it does exactly what your intuition suggests: it looks at the k closest already-labelled dishes in feature space, and lets them vote. Whichever label has the majority among those k neighbours becomes the prediction. Written as steps:
- Measure the distance from the new point to every point in the training data.
- Sort those distances from smallest to largest.
- Keep only the
ksmallest — the "nearest neighbours." - Count how many of those
kneighbours belong to each label. The label with the most votes wins.
Let's classify a genuinely new dish: Uttapam, a thick savoury pancake cooked on a griddle with a light coating of oil. Measured the same way as the rest, it comes out to oil = 5, spice = 4. Is it Fried or Steamed? We compute the distance from Uttapam (5, 4) to all six training dishes:
- to Dhokla (2, 3): √((5−2)² + (4−3)²) = √(9+1) = √10 ≈ 3.16
- to Momos (1, 4): √((5−1)² + (4−4)²) = √(16+0) = 4.00
- to Idli (1, 2): √((5−1)² + (4−2)²) = √(16+4) = √20 ≈ 4.47
- to Vada (16, 3): √((5−16)² + (4−3)²) = √(121+1) = √122 ≈ 11.05
- to Samosa (18, 4): √((5−18)² + (4−4)²) = √(169+0) = 13.00
- to Pakora (20, 5): √((5−20)² + (4−5)²) = √(225+1) = √226 ≈ 15.03
Sorted from nearest to farthest: Dhokla (3.16), Momos (4.00), Idli (4.47), Vada (11.05), Samosa (13.00), Pakora (15.03). With k = 3, the three nearest neighbours are Dhokla, Momos, and Idli — all three labelled Steamed. The vote is unanimous: 3 Steamed, 0 Fried. The classifier predicts Uttapam is Steamed, which matches how it is actually cooked. Here is the same logic as runnable Python:
import math
# Each dish's features are (oil content in g/100g, spice level 0-6)
training_data = [
((18, 4), "Fried"), # Samosa
((20, 5), "Fried"), # Pakora
((16, 3), "Fried"), # Vada
((1, 2), "Steamed"), # Idli
((2, 3), "Steamed"), # Dhokla
((1, 4), "Steamed"), # Momos
]
def distance(p1, p2):
x1, y1 = p1
x2, y2 = p2
return math.sqrt((x1 - x2) ** 2 + (y1 - y2) ** 2)
def knn_classify(new_point, data, k=3):
distances = []
for features, label in data:
d = distance(new_point, features)
distances.append((d, label))
distances.sort(key=lambda pair: pair[0]) # nearest first
nearest = distances[:k]
votes = {}
for d, label in nearest:
votes[label] = votes.get(label, 0) + 1
return max(votes, key=votes.get) # label with most votes
uttapam = (5, 4)
print(knn_classify(uttapam, training_data, k=3))
# Output: Steamed
Trace through it: knn_classify builds a list of six (distance, label) pairs, sorts it, slices the first three (Dhokla, Momos, Idli), tallies {"Steamed": 3}, and max(votes, key=votes.get) returns the key with the largest value — "Steamed". The printed output is exactly Steamed, matching our hand calculation.
A Common Misconception: "The Smallest k Is the Most Accurate"
A very natural first guess is that k = 1 — just checking the single closest match — should be the most accurate choice, since it looks at the dish that is literally most similar. This is a misconception, and it is worth correcting carefully, because it is exactly backwards in an important sense.
Real datasets, collected by real people, contain mistakes. Suppose whoever recorded our training data mistyped Dhokla's label as "Fried" instead of "Steamed" — a plausible copy-paste slip in a spreadsheet with hundreds of rows. Dhokla is still numerically at (2, 3), still the single closest point to Uttapam (5, 4) at distance 3.16, but now carries the wrong tag.
With k = 1, the classifier looks only at Dhokla, sees "Fried," and confidently — and wrongly — predicts Uttapam is Fried. One corrupted row silently breaks the whole prediction.
With k = 3, the classifier still looks at Dhokla, but also at Momos and Idli, both correctly labelled Steamed. The vote becomes 1 Fried (the bad Dhokla row) versus 2 Steamed, and the correct majority — Steamed — wins anyway. By consulting several neighbours instead of just one, the algorithm becomes resistant to a single bad or unusual data point. This is precisely why k = 1 is not automatically "best": a larger k trades a little bit of sensitivity to the single closest match for a lot of protection against noisy or mislabelled examples. The right choice of k is not "as small as possible" — it is a balance, chosen by testing.
There is a second, more mechanical reason to be careful with k: ties. For a two-class problem, an even k can produce a dead-even vote with no winner. Consider a hypothetical dish "Handvo" at (9, 3): its distance to Vada (16, 3) is √((9−16)² + (3−3)²) = √49 = 7, and its distance to Dhokla (2, 3) is √((9−2)² + (3−3)²) = √49 = 7 — exactly tied, one Fried and one Steamed. With k = 2, the vote is a flat 1–1 with no majority at all. This is why, for binary classification, practitioners deliberately choose an odd k — it guarantees a majority almost always exists.
Why the Two Features Don't Get an Equal Say
Look again at the distance formula: it squares the gap in oil content and the gap in spice level and adds them. But oil content in our data ranges from 1 to 20 (a span of 19), while spice level ranges only from 2 to 5 (a span of 3). Because the oil values are spread over a much wider range of raw numbers, a typical gap in oil content tends to be numerically larger than a typical gap in spice level — and once you square both gaps, the larger one dominates the sum even more. In effect, the classifier is currently paying far more attention to oil content than to spice, simply because of how the two features happen to be scaled, not because oil is truly more informative.
This is a genuine and well-known issue in machine learning, not a flaw specific to our toy example. The usual fix is feature scaling: rescale every feature onto a comparable range — for instance, dividing every oil value by its maximum (20) so it also lies between 0 and 1, matching a similarly rescaled spice value. Production machine learning libraries do this automatically before computing any distance. For our two-feature example the effect is modest because oil genuinely is the more decisive feature for frying versus steaming, but as you add more features with wildly different natural ranges — say, cooking time in minutes (5 to 60) alongside a spice score (0 to 6) — scaling stops being optional and becomes essential, or the feature with the biggest raw numbers will silently steamroll every other feature in the distance calculation.
Evaluating the Classifier: Confusion Matrix and Accuracy
Before trusting this classifier inside a real app, we need to check how well it performs on dishes it has never seen — not the six dishes it was built from, since correctly repeating your own training data proves nothing about how the classifier will handle a genuinely new dish. This is why machine learning always keeps a separate test set: known dishes held back purely for grading the model, never shown to it during training.
Here are four test dishes, each with a true label a human already knows, run through our k = 3 classifier:
- Kachori (oil 17, spice 5) — true label Fried. Three nearest: Samosa (1.41), Vada (2.24), Pakora (3.00) — all Fried. Predicted Fried. Correct.
- Khaman (oil 2, spice 2) — true label Steamed. Three nearest: Idli (1.00), Dhokla (1.00), Momos (2.24) — all Steamed. Predicted Steamed. Correct.
- Bhatura (oil 15, spice 1) — true label Fried. Three nearest: Vada (2.24), Samosa (4.24), Pakora (6.40) — all Fried. Predicted Fried. Correct.
- Dahi Vada (oil 7, spice 2) — true label Fried (it is deep-fried before being soaked in yogurt). Three nearest: Dhokla (5.10), Idli (6.00), Momos (6.32) — all Steamed. Predicted Steamed. Wrong.
That last error is genuinely instructive, not just an unlucky miss. Dahi Vada is fried, but once it is soaked in yogurt, the oil that was measured for our dataset is much lower than a plain, unsoaked fried snack — so in feature space it lands right next to the low-oil Steamed cluster. The classifier isn't "confused" about geometry; the geometry itself is misleading, because our chosen feature (measured oil content) doesn't fully capture the true fact that matters — how the dish was originally cooked. This is a common real lesson in machine learning: a wrong prediction is often a sign that the features don't fully capture the thing you actually care about, not simply a bug in the algorithm.
We summarise these four results in a confusion matrix, which cross-tabulates true labels against predicted labels:
- Actual Fried, Predicted Fried: 2 (Kachori, Bhatura)
- Actual Fried, Predicted Steamed: 1 (Dahi Vada)
- Actual Steamed, Predicted Steamed: 1 (Khaman)
- Actual Steamed, Predicted Fried: 0
Accuracy is simply the fraction of predictions that were correct:
Accuracy = (correct predictions) / (total predictions)
= (2 + 1) / 4
= 0.75 = 75%
75% tells us the classifier is clearly picking up a real signal — far better than the 50% you'd expect from random guessing on a two-class problem — but it also is not perfect, and the one error has an identifiable, explainable cause. That combination (a believable accuracy number plus an understandable failure case) is exactly what you want to see before shipping a model, and exactly what a single "it works!" demo on the training data would have hidden.
From a Working Function to a Deployed App
Once we're satisfied with the evaluation, the model needs to become part of a running system that vendors and users actually interact with — this is deployment. Concretely, that usually starts with wrapping the trained logic in a clean function that validates its inputs and can be called the instant new data arrives:
def predict_dish_type(oil_content, spice_level):
if oil_content < 0 or spice_level < 0:
return "Error: values cannot be negative"
return knn_classify((oil_content, spice_level), training_data, k=3)
print(predict_dish_type(14, 2))
# Output: Fried (nearest matches: Vada, Samosa, Pakora)
Tracing it: predict_dish_type(14, 2) passes the validation check, then calls knn_classify((14, 2), training_data, k=3). Distances work out to Vada 2.24, Samosa 4.47, Pakora 6.71 as the three nearest — all Fried — so the function returns "Fried", matching the comment. In a real app, this function would sit behind a screen where a vendor enters their dish's measurements, or — for a fully automated pipeline — behind a stage that estimates oil and spice from an ingredient list, and the label would appear on the menu within a second.
A few things become important the moment a model goes live that don't matter during offline experimentation:
- Latency. Notice that
knn_classifyhas no real "training" step — it stores the training data as-is and, at prediction time, computes its distance to every single training example. This style is called lazy learning. It's fine with 6 or even 6,000 training dishes, but if the canteen app eventually logs 6 million dish records, comparing a new dish against all of them on every request becomes too slow for an app that must respond instantly — a real deployment concern that specifically affects k-NN more than some other algorithms. - Concept drift. The world the model was trained on keeps changing. If air-fryer-cooked snacks become popular — crispy and fried-tasting but genuinely low in absorbed oil — they will numerically resemble the Steamed cluster and get mislabelled, exactly like Dahi Vada did. A deployed model needs to be periodically re-checked against fresh data, not trained once and forgotten.
- A feedback loop. When a vendor or user flags a wrong label — "Dahi Vada shows up as Steamed but it's fried!" — that correction is exactly the kind of new, correctly-labelled example that should be added to the training data before the next update. Production machine learning systems are never really "finished"; they are retrained as corrected data accumulates.
Where This Fits in the Bigger AI Picture
Notice the shape of everything we just did: define the problem → collect labelled examples → turn them into features → choose and train a model → evaluate it honestly on unseen data → deploy it → monitor and retrain. This sequence — often called the AI project cycle in the CBSE Artificial Intelligence curriculum (problem scoping, data acquisition, data exploration, modelling, and evaluation) — is the same skeleton behind nearly every classifier you'll encounter, no matter how sophisticated. A real photo-based food-recognition app, the kind that identifies a dish from a picture rather than from hand-measured oil and spice numbers, replaces our two features with thousands of pixel-brightness values and swaps k-NN for a deep neural network — but it still walks through data, features, training, evaluation, and deployment in exactly this order. The two-feature, arithmetic-only version you built by hand in this chapter is a small, fully transparent instance of a pipeline that scales, unchanged in its logic, all the way up to the AI systems running inside major apps.
Summary
- Classification assigns an object to one of a fixed set of labels; a classifier is a function from measured features to a label.
- Turning a dish into numbers (like oil content and spice level) places it as a point in feature space, where similar dishes cluster together.
- Euclidean distance between two feature points is just the Pythagoras theorem applied to the horizontal and vertical gaps: √((x₁−x₂)² + (y₁−y₂)²).
- k-Nearest Neighbours classifies a new point by letting its k closest labelled neighbours vote by majority.
- Smaller k is not automatically more accurate: k = 1 is vulnerable to a single noisy or mislabelled point, while a slightly larger, odd k is both more stable and avoids exact ties in a two-class problem.
- Features on very different numeric scales can unfairly dominate the distance calculation; feature scaling puts them on comparable footing.
- A model must be evaluated on a held-out test set, never on its own training data. The confusion matrix breaks predictions into correct/incorrect by class, and accuracy = correct ÷ total summarises overall performance.
- Deployment wraps the trained logic in a validated, callable function, and brings new concerns — response latency, concept drift as the real world changes, and a feedback loop that folds corrections back into future training data.
Check Your Understanding
- 1. A new snack, "Bonda," measures oil = 15, spice = 2. Using the original six training dishes and k = 3, compute the three nearest neighbours by hand and state the predicted label. (Answer: nearest neighbours are Vada at √2 ≈ 1.41, Samosa at √13 ≈ 3.61, and Pakora at √34 ≈ 5.83 — all Fried, so the prediction is Fried.)
- 2. In two or three sentences, explain why Dahi Vada was misclassified even though it genuinely is a fried dish. What does this reveal about the difference between a feature and the "true" property you actually want the model to detect?
- 3. Using the Handvo example (oil = 9, spice = 3) from this chapter, verify by direct calculation that its distances to Vada and to Dhokla are exactly equal. Why does this make k = 2 a poor choice here, and what would k = 3 do differently with this same point? (Hint: find the third-nearest neighbour and see which way the tie breaks.)
- 4. Suppose you rescale the oil feature by dividing every value by 20, so it also ranges between 0 and 1, while leaving spice level unscaled. Recompute the distance between Uttapam (5, 4) and Dhokla (2, 3) using the rescaled oil values (0.25 and 0.10) together with the original spice values. How does the new distance compare to the original 3.16, and what does that tell you about which feature now has more influence on the result?
Practice Exercises
Now it is time to practice! Complete these challenges to solidify your understanding:
- Exercise 1: Write a short program that demonstrates the core concept from this chapter. Test it with at least 3 different inputs.
- Exercise 2: Find a real-world example where building an indian food classifier: from data to deployment is used in an Indian company (like TCS, Infosys, Flipkart, or ISRO). Write a paragraph explaining the connection.
- Exercise 3: Create a mind-map connecting building an indian food classifier: from data to deployment to at least 3 other topics you have studied.