Linear Regression: Predicting Values from Data

Suppose an autorickshaw in your city charges a flat ₹30 for the first kilometre and then ₹15 for every kilometre after that. You don't need a fare meter to know the cost of an 8 km ride — you can calculate it: ₹30 for the first km, plus ₹15 × 7 for the remaining seven, giving ₹135. If you graph "distance travelled" on the x-axis and "fare" on the y-axis, every possible ride sits exactly on a straight line. You already know the equation of that line from your Class 9 mathematics — it's y = mx + c, where m is the slope (₹15 per km) and c is the y-intercept (the ₹30 base charge, roughly). Give the rule any distance, and it hands back the exact fare, with zero error, every time.

Real data is almost never that obedient. In the physical world, two quantities can be strongly related without one being a perfect, mechanical function of the other. This chapter is about what to do when the relationship is approximately linear but not exactly linear — when the points, plotted on a graph, form a scattered cloud that leans in a clear direction rather than sitting neatly on a line. The technique for finding the "best" straight line through such a cloud, and using it to predict values you haven't observed, is called linear regression — one of the oldest and most widely used tools in machine learning.

From an Exact Rule to a Messy Reality

Consider five students in a class who reported how many hours they studied for a unit test, and the marks (out of 100) they scored:

• 1 hour → 40 marks
• 2 hours → 45 marks
• 3 hours → 50 marks
• 4 hours → 60 marks
• 5 hours → 65 marks

Marks generally rise with study hours — that trend is obvious just from scanning the list. But it is not a perfect straight line. Going from 1 to 2 hours gained 5 marks; going from 3 to 4 hours gained 10 marks. If you tried to draw a single straight line through all five points, it would miss most of them by a little. Studying is not a vending machine that dispenses marks on a fixed exchange rate — a hundred other things affect a student's score: how well they slept, whether the questions matched what they revised, sheer luck on that day. Linear regression doesn't pretend those factors don't exist. Instead, it asks a more modest, more useful question: of all the possible straight lines, which one comes closest, overall, to this scattered data — and can we use that line to make a reasonable prediction?

In machine learning vocabulary, "hours studied" is called the feature (input, usually written x), and "marks scored" is the target (output, y) — the quantity we're trying to predict. The five (hours, marks) pairs are our training data: the examples the model learns from before it is asked to predict for a new, unseen input.

The Line of Best Fit

To compare candidate lines, we need a way to measure how badly a given line misses the data. For any line y = mx + c and any data point (xᵢ, yᵢ), the line predicts a value ŷᵢ = m·xᵢ + c (read "y-hat", the model's guess) at that x. The difference between what actually happened and what the line predicted is called the residual, or error:

residual = yᵢ − ŷᵢ = actual − predicted

A good line keeps these residuals small across all the points. But you cannot just add up the raw residuals and call the sum "total error" — some residuals are positive (the line underestimated) and some are negative (the line overestimated), and they cancel each other out. A line that misses one point by +20 and another by −20 would score a perfect "0 total error" by this measure, even though it's clearly a bad fit for those two points. This is exactly the trap many learners fall into, so it's worth naming directly: summing raw errors is not a valid way to judge a line, because errors of opposite sign cancel.

The fix used in linear regression is to square each residual before adding it up. Squaring makes every term positive (or zero), so cancellation is impossible, and it also punishes large misses more heavily than small ones — a residual of 4 contributes 16 to the total, while two residuals of 2 each contribute only 4 + 4 = 8. The line chosen by linear regression is the one that makes this sum of squared residuals as small as possible. This is why the standard method is called least squares regression, and the resulting line is called the least-squares line or line of best fit.

It turns out — and you can verify this with basic algebra once you reach calculus in later grades — that there is a direct formula for the slope and intercept of this best line, no trial and error required:

m = Σ(xᵢ − x̄)(yᵢ − ȳ) / Σ(xᵢ − x̄)²

c = ȳ − m·x̄

Here x̄ (x-bar) and ȳ (y-bar) are the mean (average) of the x-values and y-values, and Σ means "sum over all the data points." It looks intimidating written like that, so let's compute it by hand on our five-student dataset and watch every piece fall into place.

Computing the Best-Fit Line by Hand

Step 1 — find the means. The hours are 1, 2, 3, 4, 5, which sum to 15, so x̄ = 15 / 5 = 3. The marks are 40, 45, 50, 60, 65, which sum to 260, so ȳ = 260 / 5 = 52.

Step 2 — find each point's deviation from the mean, for both x and y:

• x = 1: deviation = 1 − 3 = −2  |  y = 40: deviation = 40 − 52 = −12
• x = 2: deviation = 2 − 3 = −1  |  y = 45: deviation = 45 − 52 = −7
• x = 3: deviation = 3 − 3 = 0   |  y = 50: deviation = 50 − 52 = −2
• x = 4: deviation = 4 − 3 = 1   |  y = 60: deviation = 60 − 52 = 8
• x = 5: deviation = 5 − 3 = 2   |  y = 65: deviation = 65 − 52 = 13

Step 3 — multiply each pair of deviations, and separately square each x-deviation:

• (−2)(−12) = 24   and   (−2)² = 4
• (−1)(−7) = 7    and   (−1)² = 1
• (0)(−2) = 0      and   (0)² = 0
• (1)(8) = 8       and   (1)² = 1
• (2)(13) = 26    and   (2)² = 4

Step 4 — sum both columns. The products sum to 24 + 7 + 0 + 8 + 26 = 65. The squared x-deviations sum to 4 + 1 + 0 + 1 + 4 = 10.

Step 5 — plug into the formulas: m = 65 / 10 = 6.5, and c = ȳ − m·x̄ = 52 − 6.5 × 3 = 52 − 19.5 = 32.5.

So the least-squares line for this class is:

Predicted Marks = 6.5 × Hours + 32.5

Read the two numbers as a story about the data: the slope, 6.5, says that on average, each additional hour of study is associated with about 6.5 extra marks. The intercept, 32.5, is what the line predicts at zero hours of study — not a claim that an unprepared student scores exactly 32.5, but the mathematical starting point of the trend line.

Verifying with Code

The formula translates directly into a short program. Nothing here is a machine-learning library trick — it's the same five steps you just did by hand, run in a loop:

hours = [1, 2, 3, 4, 5]
marks = [40, 45, 50, 60, 65]

n = len(hours)
mean_x = sum(hours) / n
mean_y = sum(marks) / n

numerator = 0
denominator = 0
for i in range(n):
    numerator += (hours[i] - mean_x) * (marks[i] - mean_y)
    denominator += (hours[i] - mean_x) ** 2

m = numerator / denominator
c = mean_y - m * mean_x

print("slope m =", m)
print("intercept c =", c)

def predict(x):
    return m * x + c

print("Predicted marks for 6 hours:", predict(6))

Trace it exactly as Python would: mean_x becomes 3.0 and mean_y becomes 52.0. The loop runs five times, accumulating numerator as 24, then 31, then 31, then 39, then finally 65; denominator accumulates as 4, 5, 5, 6, then 10. After the loop, m = 65 / 10 = 6.5 and c = 52.0 − 6.5 × 3.0 = 32.5. The program prints slope m = 6.5, intercept c = 32.5, and finally, since predict(6) = 6.5 × 6 + 32.5 = 39 + 32.5 = 71.5, it prints Predicted marks for 6 hours: 71.5. Every number matches the hand calculation exactly — the code isn't doing anything mysterious, it's just doing the arithmetic faster.

Seeing It on a Graph

Plotting the five points together with this line makes the "least squares" idea visible rather than abstract. Notice that the line doesn't touch every point — at 3 hours it predicts 52 marks but the student actually scored 50, a residual of −2; at 4 hours it predicts 58.5 but the student scored 60, a residual of +1.5. Those small vertical gaps are exactly the residuals the formula was built to minimize, shown below as dashed segments.

One more thing worth checking by hand: add up all five residuals for this line — (40−39) + (45−45.5) + (50−52) + (60−58.5) + (65−65) = 1 − 0.5 − 2 + 1.5 + 0 = 0. That's not a coincidence. It's a genuine mathematical property of the least-squares line: the positive and negative residuals always balance out to exactly zero. This is a useful sanity check whenever you compute a regression line by hand — if your residuals don't sum to (approximately) zero, you've made an arithmetic mistake.

Is This Really the Best Line? Comparing Candidates

It's reasonable to be skeptical of the formula and check it against a plausible alternative. Suppose a classmate eyeballs the same scatter plot and guesses the line y = 7x + 30 instead — a fairly sensible-looking guess. Whose line actually fits better? We compare using the sum of squared residuals, exactly as the least-squares method defines "best":

hours = [1, 2, 3, 4, 5]
marks = [40, 45, 50, 60, 65]

def sse(m, c):
    total = 0
    for x, y in zip(hours, marks):
        prediction = m * x + c
        error = y - prediction
        total += error ** 2
    return total

print("Best-fit line (m=6.5, c=32.5):", sse(6.5, 32.5))
print("Guess line       (m=7, c=30):", sse(7, 30))

For the least-squares line, the predictions are 39, 45.5, 52, 58.5, 65, giving residuals of 1, −0.5, −2, 1.5, and 0, whose squares (1, 0.25, 4, 2.25, 0) sum to 7.5. For the guessed line, the predictions are 37, 44, 51, 58, 65, giving residuals of 3, 1, −1, 2, and 0, whose squares (9, 1, 1, 4, 0) sum to 15. The computed line's total squared error (7.5) is exactly half the guessed line's (15) — confirming, with real numbers rather than blind trust, that the formula genuinely finds a better line than a reasonable human guess. Also notice that the guessed line's raw residuals sum to 3+1−1+2+0 = 5, not zero — reinforcing that the "residuals sum to zero" property is special to the least-squares line, not something every line has.

Using the Model to Predict — and Its Limits

Once you have m and c, predicting for a new input is just substitution. If a sixth student reports studying 6 hours, the model predicts 6.5 × 6 + 32.5 = 71.5 marks. This is called interpolation-adjacent prediction when the new x is close to the range you trained on (our data ran from 1 to 5 hours, so 6 is a small, reasonable step beyond it).

But push the model further and it breaks down. If you plug in 20 hours studied, the formula obediently returns 6.5 × 20 + 32.5 = 162.5 marks — a meaningless answer, since marks are capped at 100 and no one studies 20 hours for one test anyway. This is the danger of extrapolation: a linear model only knows the trend within the range of data it was trained on. Outside that range, there is no guarantee the real-world relationship stays a straight line — it might flatten out (diminishing returns from over-studying), curve, or simply stop making sense. A trustworthy prediction stays close to the range of the training data; a distant extrapolation is a guess dressed up as a calculation.

Two Common Misconceptions

Misconception 1: "The regression line has to pass through at least one of the data points." It doesn't. Look again at our five students — the line y = 6.5x + 32.5 misses every single one of them by at least a small amount, and it is still the mathematically best line, because it minimizes the total squared miss across all five points simultaneously. A line is judged by its overall fit to the whole dataset, not by how many individual points it happens to touch.

Misconception 2: "If two things are linearly related, one must be causing the other." This is one of the most important cautions in all of statistics. A classic example: in many places, months with higher ice-cream sales also see more reported drowning incidents. A regression line fit to (ice-cream sales, drownings) would show a real, strong upward trend. But eating ice-cream does not cause drowning. Both are driven by a third factor — hot weather, which independently increases ice-cream purchases and the number of people swimming. This is called a confounding variable. Linear regression can tell you that two quantities move together and let you predict one from the other — it cannot, by itself, tell you that one causes the other. Whenever you build a regression model from real data, ask what else might be driving both variables before claiming a cause.

Where the Name Comes From

The term "regression" has a specific historical origin. In the 1880s, the scientist Francis Galton studied the heights of parents and their adult children and noticed that very tall parents tended to have children who were tall, but usually not as extreme as the parents — and very short parents tended to have children closer to average height than themselves. Galton called this pull toward the average "regression toward mediocrity" (we'd now say "regression to the mean"). The line-fitting technique he used to study this effect kept the name "regression" even though modern uses of the method — like predicting marks from study hours — have nothing to do with heights or extremes drifting toward an average. The name stuck; the technique outgrew its original, narrower meaning.

Check Your Understanding

1. A shop records the number of hours it stays open beyond 9 AM (x) each day and the day's sales in ₹ hundreds (y): x = 2, 4, 6, 8, 10 gives y = 40, 55, 75, 85, 110. Compute the least-squares slope and intercept by hand.
2. Using the line you just found, predict the sales for 12 hours open. Is this prediction trustworthy? Why or why not?
3. Explain in your own words why we square the residuals instead of simply adding them up.
4. True or false, with a reason: "A well-fit regression line will pass through most or all of its training data points."
5. A researcher notices that towns with more fire stations also tend to report more fire damage per year, and fits a regression line showing this trend. A newspaper reports: "Fire stations cause more fire damage." What is wrong with this conclusion, and what confounding variable might actually explain the pattern?

Answers. (1) x̄ = 6, ȳ = 73. Deviations in x: −4, −2, 0, 2, 4; in y: −33, −18, 2, 12, 37. Products: 132, 36, 0, 24, 148, summing to 340. Squared x-deviations: 16, 4, 0, 4, 16, summing to 40. So m = 340/40 = 8.5 and c = 73 − 8.5×6 = 22, giving y = 8.5x + 22. (2) At x = 12: 8.5×12 + 22 = 124 (₹12,400). This is a modest extrapolation, just two hours beyond the training range (2–10), so it's a reasonable estimate but should be treated with more caution than a prediction inside the observed range — a shop physically cannot stay open indefinitely, and sales patterns may not stay linear that far out. (3) Raw residuals can be positive or negative and cancel when summed, so a line could have a "total error" of zero while still missing every point badly; squaring makes every contribution positive, so cancellation is impossible and large misses are penalized more than small ones. (4) False — the least-squares line is chosen to minimize total squared error across all points and typically touches none of them exactly, as shown in the worked marks-vs-hours example, where every one of the five points had a nonzero residual. (5) Correlation isn't causation; a likely confounding variable is population density or town size — larger, denser towns build more fire stations to serve their larger population, and also simply have more buildings and activity that can catch fire, producing both trends independently of each other.

Summary

Linear regression fits a straight line, y = mx + c, to data that trends in a roughly linear way but doesn't sit exactly on any line — the everyday situation for almost all real measurements, unlike an exact formula such as a fixed autorickshaw tariff. The "best" line is defined as the one minimizing the sum of squared residuals (actual minus predicted, squared and added up), because squaring prevents positive and negative errors from cancelling and penalizes large misses more heavily. The slope and intercept of this best line can be computed directly from the data's means and deviations using m = Σ(xᵢ−x̄)(yᵢ−ȳ) / Σ(xᵢ−x̄)² and c = ȳ − m·x̄ — no guessing required, and the same five steps translate directly into a short program. Once fitted, the line can predict y for new x-values, but only reliably within or near the range of the training data; going far outside that range is extrapolation, and the linear trend has no guarantee of holding there. A well-fit regression line generally misses every individual data point by a little, yet its residuals always sum to exactly zero — and fitting such a line, however well, only reveals association between two quantities, never proof that one causes the other.

Think About It

Think about this: How would you explain linear regression: predicting values from data to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.

Practice Exercises

Now it is time to practice! Complete these challenges to solidify your understanding:

• Exercise 1: Write a short program that demonstrates the core concept from this chapter. Test it with at least 3 different inputs.
• Exercise 2: Find a real-world example where linear regression: predicting values from data is used in an Indian company (like TCS, Infosys, Flipkart, or ISRO). Write a paragraph explaining the connection.
• Exercise 3: Create a mind-map connecting linear regression: predicting values from data to at least 3 other topics you have studied.