Introduction to Machine Learning with Python

A Program That Refuses to Follow Fixed Rules

Suppose your friend asks you to write a Python function that predicts a student's marks in a test based on how many hours they studied. You have five real results from your class to go on: a student who studied 1 hour scored 35, one who studied 2 hours scored 45, one who studied 3 hours scored 55, one who studied 4 hours scored 60, and one who studied 5 hours scored 75. Your first instinct, as a programmer, is to write rules.

def predict_marks(hours):
    if hours == 1:
        return 35
    elif hours == 2:
        return 45
    elif hours == 3:
        return 55
    elif hours == 4:
        return 60
    elif hours == 5:
        return 75
    else:
        return None  # no idea what happens here

This function works perfectly for exactly five inputs and is useless for everything else. Ask it about 2.5 hours, or 6 hours, or 3.7 hours, and it shrugs with None. You could keep adding elif lines forever, but you would only ever cover the exact cases you already know the answer to. That is the core limitation of traditional, rule-based programming: a human has to sit down and hand-write every rule, and the program can never do better than the rules it was given.

Machine learning flips this process around. Instead of a human writing the rule, the computer looks at the examples and works out a general rule by itself — one that can make a sensible guess even for hours it has never seen before, like 2.5 or 6. This chapter builds that idea from the ground up, using real Python code you can run yourself, and ends with the same dataset being used for two genuinely different kinds of prediction: estimating a number (regression) and choosing a category (classification).

What Exactly Is "Learning" Here?

In traditional programming, you supply the computer with rules and data, and it produces answers. In machine learning, you supply the computer with data and the already-known answers, and it produces the rules. This is the single most important sentence in this chapter, so it is worth sitting with: a machine learning model is not a database of memorized answers — it is a compact mathematical rule, discovered from examples, that generalizes to new inputs.

For our study-hours example, the "rule" the computer needs to discover is very simple: as hours studied goes up, marks tend to go up too, in a roughly straight-line way. If we can describe that straight line precisely — its steepness and its starting point — we have a working model. Finding that line is exactly what the algorithm called linear regression does, and it is the first machine learning algorithm we will build, first by hand and then in Python.

Our Dataset: Hours Studied vs Marks Scored

Here are the five data points we will use throughout this section:

• Student A: 1 hour studied → 35 marks
• Student B: 2 hours studied → 45 marks
• Student C: 3 hours studied → 55 marks
• Student D: 4 hours studied → 60 marks
• Student E: 5 hours studied → 75 marks

Plotted on a graph, with hours studied along the bottom and marks up the side, these five points roughly climb upward and to the right, but not perfectly — they do not sit exactly on any single straight line. That "not quite exact" behavior is completely normal in real data (some students revise more efficiently than others, some get lucky with the questions asked), and machine learning does not need every point to fit perfectly. It needs to find the single straight line that fits the points best overall, even though it will miss most of them slightly.

Finding the Best-Fit Line by Hand

Every straight line can be written as y = m·x + c, where x is the input (hours studied), y is the predicted output (marks), m is the slope (how many extra marks you gain per extra hour studied), and c is the intercept (the predicted marks when hours studied is zero). "Training" a linear regression model simply means calculating the values of m and c that make the line fit the data best.

The standard method, called least squares, picks the line that minimizes the total squared distance between the actual points and the line. The formula looks intimidating at first, but it only uses arithmetic you already know: averages, subtraction, multiplication, and division.

Step 1 — find the average of the x-values (hours) and the average of the y-values (marks):

• average of hours = (1 + 2 + 3 + 4 + 5) / 5 = 15 / 5 = 3
• average of marks = (35 + 45 + 55 + 60 + 75) / 5 = 270 / 5 = 54

Step 2 — for each student, find how far their hours and marks are from these averages, and multiply the two differences together:

• Student A: (1 − 3) × (35 − 54) = (−2) × (−19) = 38
• Student B: (2 − 3) × (45 − 54) = (−1) × (−9) = 9
• Student C: (3 − 3) × (55 − 54) = (0) × (1) = 0
• Student D: (4 − 3) × (60 − 54) = (1) × (6) = 6
• Student E: (5 − 3) × (75 − 54) = (2) × (21) = 42

Adding these up: 38 + 9 + 0 + 6 + 42 = 95.

Step 3 — for each student, square how far their hours are from the average, and add those up:

• (−2)² + (−1)² + (0)² + (1)² + (2)² = 4 + 1 + 0 + 1 + 4 = 10

Step 4 — divide the two totals to get the slope: m = 95 / 10 = 9.5. This means each extra hour of study is worth, on average, 9.5 extra marks in this small dataset.

Step 5 — find the intercept using c = average(y) − m × average(x): c = 54 − 9.5 × 3 = 54 − 28.5 = 25.5.

So the model's full rule is: predicted marks = 9.5 × hours + 25.5. Let's check it against a student the line does not pass through exactly, Student D, who studied 4 hours: 9.5 × 4 + 25.5 = 38 + 25.5 = 63.5, but Student D actually scored 60. The model is off by 3.5 marks for this student — and that is fine. Least squares does not promise a perfect fit for every point; it promises the smallest possible total squared error across all points combined. That is a crucial distinction from rule-based programming, where every case either matches a rule exactly or falls through to "no answer."

Teaching Python to Find the Same Line

Doing this arithmetic by hand for five points is manageable; doing it for five thousand points is not. This is exactly the job of the scikit-learn library, which is the standard machine learning toolkit used with Python.

from sklearn.linear_model import LinearRegression
import numpy as np

hours = np.array([1, 2, 3, 4, 5]).reshape(-1, 1)
marks = np.array([35, 45, 55, 60, 75])

model = LinearRegression()
model.fit(hours, marks)

print("Slope (m):", model.coef_[0])
print("Intercept (c):", model.intercept_)

predicted = model.predict([[6]])
print("Predicted marks for 6 hours:", predicted[0])

Tracing through this line by line: hours is reshaped into a column (scikit-learn always expects a 2D array of inputs, even when there is only one feature per student, which is why we call .reshape(-1, 1)). model.fit(hours, marks) is the actual "learning" step — internally, scikit-learn runs the exact same least-squares calculation we just did by hand. When you run this, the output is:

Slope (m): 9.5
Intercept (c): 25.5
Predicted marks for 6 hours: 82.5

These match our hand calculation exactly, which is a good way to sanity-check that you understand what the library is doing rather than treating it as a black box. (On some systems you might see a value like 9.500000000000002 instead of a clean 9.5 — this is ordinary floating-point rounding inside the computer's arithmetic, not a bug, and it means the same thing.) For 6 hours studied, a value the model never saw during training, it predicts 82.5 marks by simply continuing the line: 9.5 × 6 + 25.5 = 82.5.

The Misconception: "Perfect Accuracy on Known Data Means a Great Model"

Many CBSE students, used to scoring 100% by memorizing an answer key, assume the best possible model is one that predicts every training point exactly. This is actually a warning sign, not a success, and it has a name: overfitting.

Imagine building a wildly curved model, instead of a straight line, that bends itself to pass through all five of our data points exactly — zero error on Students A through E. That model has essentially memorized the answer key. Give it a sixth student who studied 2.5 hours, and the curve might do something absurd, like predicting 20 marks, because it was never forced to find a sensible general trend — it just contorted itself to match the specific points it was shown. The straight line, by contrast, is wrong on every single training student by a few marks, yet it generalizes far better to new students, because it captured the real underlying pattern (more study time helps, roughly linearly) instead of the noise (the small random variations between individual students).

This is why real machine learning projects always keep some data hidden from the model during training, called test data, and check accuracy only on that unseen portion. A model that scores 100% on data it was trained on but poorly on test data has memorized, not learned — the machine learning equivalent of memorizing last year's board exam paper instead of understanding the chapter.

A Different Kind of Question: Classification

Predicting marks is a regression problem — the output is a number that can take many values. But not every question has a numeric answer. Suppose instead we want to predict whether a student will Pass or Fail, based on their attendance percentage. Now there are only two possible outputs, not a continuous range, so a straight line predicting "marks" no longer makes sense. This is a classification problem, and it needs a different kind of algorithm.

Here is our new dataset, six students with their attendance percentage and their result:

• 40% attendance → Fail
• 50% attendance → Fail
• 60% attendance → Fail
• 75% attendance → Pass
• 85% attendance → Pass
• 95% attendance → Pass

K-Nearest Neighbors: Judging by the Company You Keep

One of the simplest and most intuitive classification algorithms is called K-Nearest Neighbors, usually abbreviated KNN. The idea is exactly what it sounds like: to predict the outcome for a new student, look at the k existing students who are most similar to them (here, "similar" means closest in attendance percentage), and let those neighbors vote. Whichever outcome has the majority among the neighbors becomes the prediction.

Let's classify a new student with 68% attendance, using k = 3 (look at the 3 nearest neighbors). First, calculate how far 68 is from every known attendance value:

• |68 − 40| = 28
• |68 − 50| = 18
• |68 − 60| = 8
• |68 − 75| = 7
• |68 − 85| = 17
• |68 − 95| = 27

Sorting these distances from smallest to largest, the three closest students are: 75% (distance 7, Pass), 60% (distance 8, Fail), and 85% (distance 17, Pass). Among these three neighbors, Pass appears twice and Fail appears once, so the majority vote is Pass — even though the new student's attendance (68%) is numerically closer to the Fail-labeled 60% than the Pass-labeled 75% and 85% combined, the vote among the full trio still swings to Pass because two of the three nearest neighbors are Pass.

Here is the same logic written as Python code, using scikit-learn's built-in KNN classifier:

from sklearn.neighbors import KNeighborsClassifier
import numpy as np

attendance = np.array([40, 50, 60, 75, 85, 95]).reshape(-1, 1)
result = np.array([0, 0, 0, 1, 1, 1])  # 0 = Fail, 1 = Pass

knn = KNeighborsClassifier(n_neighbors=3)
knn.fit(attendance, result)

new_student = [[68]]
prediction = knn.predict(new_student)
print("Prediction for 68% attendance:", "Pass" if prediction[0] == 1 else "Fail")

Running this prints Prediction for 68% attendance: Pass, matching our hand calculation exactly. Notice something important about the shape of this code compared to the regression example earlier: we import a different class (KNeighborsClassifier instead of LinearRegression), but the overall pattern — create the model, call .fit() with inputs and known answers, call .predict() on new data — is identical. This consistent pattern, called an API, is one reason scikit-learn is used so widely: once you understand .fit() and .predict() for one algorithm, you already understand the basic usage of dozens of others.

One subtlety worth naming: the choice of k matters. If we had used k = 1 for our 68% student, the single nearest neighbor is 75% (distance 7), which is a Pass, so the prediction would still be Pass. But if we had used k = 5, the five nearest neighbors would be 75, 60, 85, 50, and 95 (distances 7, 8, 17, 18, 27), giving three Pass votes (75, 85, 95) against two Fail votes (60, 50) — still Pass, but by a narrower margin. A poorly chosen k (say, k equal to the entire dataset) would just predict whatever the overall majority class is for every single new input, ignoring the input's actual value completely — which is why choosing k thoughtfully, usually by testing a few values against held-out data, is itself part of building a good model.

The Three Broad Families of Machine Learning

Both examples in this chapter belong to supervised learning: we always had the correct answers (marks, or Pass/Fail) already available for our training examples, and the algorithm's job was to learn the mapping from input to known output. Supervised learning splits further into the two problem types we just saw: regression (predicting a number, like marks) and classification (predicting a category, like Pass/Fail).

There are two other broad families you should know the shape of, even though we will not build them in this chapter. Unsupervised learning is given data with no known answers at all, and its job is to find structure on its own — for example, grouping our students into clusters of similar study habits without ever being told what the "correct" groups are. Reinforcement learning is different again: an agent takes actions in an environment and learns from rewards and penalties over time, the way a game-playing program improves by repeatedly playing and being scored, rather than by being shown labeled examples upfront. For your CBSE board exam, the detail worth remembering precisely is the difference between regression and classification within supervised learning, since that distinction is the one you will be asked to apply to given scenarios.

Why This Matters Beyond the Classroom: Correlation Is Not Causation

Our attendance model found a genuine pattern in this particular small dataset: students with higher attendance tended to pass. But notice what the model actually learned — a correlation between two numbers, nothing more. It has no concept of why attendance and passing are related. In the real world, low attendance can be caused by many different things: disengagement, yes, but also long commutes, family responsibility, or health issues — and treating "low attendance" as if it directly causes failure, rather than being one symptom among several possible underlying causes, can lead a real system to unfairly penalize students who are already facing hardship, rather than helping them.

This exact failure pattern shows up in real deployed systems: an admissions or loan-approval model trained only on a narrow slice of past applicants can learn to associate an unrelated factor (like postal code or school name) with outcomes, and then apply that pattern unfairly to new applicants who don't resemble the original training population. The technical fix (testing models on diverse, representative data before trusting their predictions) and the ethical responsibility (asking whether a correlation is strong enough, and fair enough, to justify a real decision about a real person) are both part of building machine learning systems responsibly, and both start with the same habit we practiced by hand in this chapter: knowing exactly what your model is and is not doing, instead of trusting it blindly because the output looks precise.

Check Your Understanding

• Using the hand-derived rule marks = 9.5 × hours + 25.5, what does the model predict for a student who studied 8 hours? Does this reveal any risk in trusting a linear model far outside the range of its training data (1 to 5 hours)?
• A classmate builds a curve that scores 100% accuracy on all five training students but performs badly on new students. Name the phenomenon and explain, in your own words, why a straight line with nonzero error on every training point can still be the better model.
• For the attendance dataset, use KNN with k = 5 to classify a new student with 55% attendance. List the five nearest neighbors with their distances and the final vote.
• Is "predicting a student's final board exam percentage from their unit test scores" a regression problem or a classification problem? Justify your answer in one sentence.
• Explain, using the attendance example, why a model finding that "low attendance is associated with failing" is not the same as proving "low attendance causes failing."

Summary

Machine learning replaces hand-written rules with rules discovered from data. A linear regression model finds the straight line (defined by slope m and intercept c) that minimizes total squared error across all training examples, computed here by hand using averages and verified with scikit-learn's LinearRegression. A K-Nearest Neighbors classifier instead predicts a category by letting the k most similar known examples vote, computed here using simple distance arithmetic and verified with scikit-learn's KNeighborsClassifier. Both algorithms share the same core scikit-learn pattern: .fit() to learn from labeled examples, .predict() to apply that learning to new inputs. The most important idea to carry forward is that a good model captures the general trend rather than memorizing every training point exactly (overfitting), and that even a model with strong accuracy on past data must be checked for whether its patterns are fair and causally meaningful before it is trusted to make decisions about real people.