Maximum Likelihood Estimation (MLE) Basics

The IRCTC Waitlist Problem

You've booked a waitlisted train ticket on IRCTC. You want to know: what is the probability that a waitlisted ticket like yours eventually confirms? Nobody hands you this number — Indian Railways doesn't publish it for your specific route and quota. But you have something almost as good: history. You pull up records for the last 20 waitlisted tickets booked in your exact quota and route, and you find that 14 of them confirmed and 6 didn't.

What's your best guess for the true confirmation probability, call it p? Almost everyone's instinct is the same: 14 out of 20, so p ≈ 0.7. That instinct is correct — but "it feels right" is not a proof, and a 16-year-old preparing for JEE-level probability should be able to say exactly why 0.7 beats every other candidate value, including 0.65, 0.72, or 0.75. Maximum Likelihood Estimation (MLE) is the precise mathematical machinery that turns "14/20 feels right" into a theorem. That is what this chapter builds, from the ground up.

A Number Before a Formula: Testing Candidate Values

Before writing any formula, let's just test some guesses. Suppose the true confirmation probability were exactly p = 0.5 (a coin-flip). Then the probability of observing exactly 14 confirmations out of 20 tickets is given by the binomial probability formula you already know from Class 11-12 probability:

P(14 confirmations out of 20 | p) = C(20,14) · p^14 · (1-p)^6

Plugging in a few candidate values of p and computing this quantity gives a very telling table (C(20,14) = 38760 in every row, since that combinatorial factor doesn't depend on p at all):

p = 0.5  →  L(p) = 38760 × 0.5^14 × 0.5^6   ≈ 0.0370
p = 0.6  →  L(p) = 38760 × 0.6^14 × 0.4^6   ≈ 0.1244
p = 0.7  →  L(p) = 38760 × 0.7^14 × 0.3^6   ≈ 0.1916
p = 0.8  →  L(p) = 38760 × 0.8^14 × 0.2^6   ≈ 0.1091

Look at what happened: as the candidate value climbed from 0.5 to 0.7, the computed number kept rising. Then, going from 0.7 to 0.8, it fell back down. Among every value we could plug in for p, 0.7 makes the observed data (14 confirmations out of 20) the most probable outcome. No other single value of p gives the data as high a "score." This scoring function — the probability of the fixed, already-observed data, written as a function of the unknown parameter — is called the likelihood function, and the value of p that maximizes it is the maximum likelihood estimate, written p̂ (read "p-hat").

Probability vs. Likelihood: The Flip That Confuses Everyone

This is the single most important distinction in the whole chapter, and it's exactly where most students — and most textbook explanations — get sloppy. The formula

P(data | parameter)

can be read in two completely different directions, depending on which side you treat as fixed and which side you let vary.

• Probability: fix the parameter p (say, you're told the true confirmation rate is exactly 0.7), and ask how likely different data outcomes are. "If p = 0.7, what's the chance of seeing 14, or 10, or 20 confirmations out of 20 tickets?" Here p is known and the data varies — and if you summed P(k confirmations) over every possible value of k from 0 to 20, you would get exactly 1, because probabilities over outcomes must add to 1.
• Likelihood: fix the data (you already observed 14 out of 20 — that's not changing), and ask how plausible different parameter values are in light of that fixed data. "Given that I saw 14 out of 20, how does the plausibility score change as I try different values of p?" Here the data is locked and the parameter varies — and, as you'll verify shortly, this function does not add up to 1 when you sum or integrate it over all possible values of p. It isn't a probability distribution over p at all.

Same formula, same numbers plugged in — completely different question being asked. That's why statisticians write L(p) instead of P(p) when the parameter is the thing varying: it's a reminder that you're reading the formula in the "likelihood direction," not the "probability direction."

Building the General Likelihood Function

Now let's generalize past the specific numbers 14 and 20. Suppose you observe n independent waitlisted tickets (independent meaning one ticket's outcome doesn't affect another's — a reasonable assumption when tickets belong to different passengers), and k of them confirm. Each individual ticket is a Bernoulli trial: it confirms with probability p and fails to confirm with probability (1 − p). The likelihood function for the whole dataset is:

L(p) = C(n,k) · p^k · (1-p)^(n-k),  for 0 ≤ p ≤ 1

This is the binomial probability mass function — nothing new there. What is new is the mental shift: n and k are now the fixed, known numbers (they came from your data), and p is the variable you're solving for. We want the value of p in [0, 1] that makes L(p) as large as possible.

Finding the Peak: Log-Likelihood and Calculus

Testing a handful of candidate values, as we did above, tells you roughly where the peak is but not its exact location — and for messier problems, guess-and-check simply won't scale. We need calculus to find the exact maximum. If differentiation is new territory for you (full rules arrive in Class 11), here is everything you need, self-contained: the derivative of ln(x) with respect to x is 1/x, and at the peak of a smooth hump-shaped curve, the slope (derivative) is exactly zero — the curve is momentarily flat right at the top.

Maximizing L(p) directly is awkward because it's a product of two things raised to powers. So we use a standard trick: since ln(x) is a strictly increasing function, whatever value of p maximizes L(p) also maximizes ln L(p) — taking a logarithm never changes where the peak sits, only how tall it looks. This "log-likelihood" trick converts a messy product into a sum, which is far easier to differentiate:

ln L(p) = ln C(n,k) + k·ln(p) + (n-k)·ln(1-p)

The first term doesn't involve p at all, so its derivative is zero — it plays no role in locating the maximum. Differentiating the rest, term by term, using d/dp[ln p] = 1/p and the chain rule for d/dp[ln(1-p)] = -1/(1-p):

d/dp [ln L(p)] = k/p - (n-k)/(1-p)

Set this equal to zero and solve for p — this is the step that pins down the exact peak:

k/p = (n-k)/(1-p)
k(1-p) = p(n-k)
k - kp = np - kp
k = np
p̂ = k/n

Notice the −kp term appears on both sides and cancels cleanly, leaving the beautifully simple result p̂ = k/n — exactly the "intuitive" answer of 14/20 = 0.7 you guessed at the very start, now derived rather than assumed. To confirm this critical point is a maximum and not a minimum or a saddle, check the second derivative: d²/dp²[ln L(p)] = −k/p² − (n−k)/(1−p)², which is negative for every p strictly between 0 and 1 (both terms are negative, since squares are positive and we subtract). A negative second derivative means the curve is concave — bending downward — everywhere on its domain, so the single point where the slope is zero must be the one and only maximum.

Visualizing the Likelihood Function

The following diagram plots L(p) across the full range of candidate values for our IRCTC example (n = 20, k = 14), computed the same way as the table above. Watch how the curve rises, peaks sharply at exactly p = 0.70, and falls again — precisely as the calculus predicted.

The curve is not symmetric — it leans, because p = 0.70 sits closer to the upper boundary of 1 than to the lower boundary of 0, and the binomial formula respects that boundary. This asymmetry is itself informative: likelihood curves for parameters bounded on one or both sides are rarely perfect bell shapes, unlike the normal distribution you meet later in this chapter.

Common Misconception: "L(0.7) Is the Probability That p = 0.7"

Read that heading again, because this exact sentence is what most students silently believe after seeing a likelihood curve for the first time — and it is false. L(0.7) ≈ 0.1916 is not "the probability that the true confirmation rate equals 0.7." It is the probability of observing our specific data (14 out of 20) if the true rate happened to be 0.7. Two things confirm this isn't a probability distribution over p:

• It doesn't integrate to 1. If you added up (technically, integrated) L(p) across every value of p from 0 to 1, you would not get 1 — you'd get some other number entirely. A genuine probability distribution over a continuous variable must integrate to exactly 1; this curve has no such obligation because it was never built to satisfy that rule.
• It answers a different question. "How probable is this data, assuming p takes this value?" is not the same question as "how probable is it that p takes this value, given this data?" Confusing the two is a well-known logical error (sometimes called the "prosecutor's fallacy" in legal statistics) — mixing up P(evidence | hypothesis) with P(hypothesis | evidence).

Assigning an actual probability to a parameter value — "there's a 60% chance the true rate is above 0.65" — is what Bayesian statistics does, and it requires an extra ingredient MLE deliberately avoids: a prior distribution reflecting your belief about p before seeing any data, which combines with the likelihood (via Bayes' theorem — the same theorem in your CBSE probability syllabus) to produce a posterior distribution that genuinely does integrate to 1. Maximum likelihood estimation takes the more minimalist route: it treats p as an unknown but fixed constant of nature, refuses to assign it a probability, and simply reports the single value that makes the observed data least surprising.

A Second Worked Example: MLE for the Mean of a Normal Distribution

The waitlist problem involved a discrete outcome (confirmed or not) and a discrete count. MLE works identically for continuous, real-valued data — and applying it here produces a satisfying surprise: it recovers a formula you already know.

Suppose you measure the diameters of n ball bearings coming off a production line — say, at a components supplier in Pune making parts for auto manufacturers. You model each measurement xᵢ as drawn independently from a Normal distribution with unknown mean μ and known variance σ² (variance measures the manufacturing process's inherent spread, which the factory has already characterized from years of data; only the mean — the process's current center point — is unknown today). The probability density for a single measurement is:

f(x_i ; μ) = 1/√(2πσ²) · exp( -(x_i - μ)² / (2σ²) )

Because the n measurements are independent, their joint density — the likelihood of seeing this entire dataset — is the product of the individual densities:

L(μ) = ∏(i=1 to n) f(x_i ; μ)

Products of exponentials are unpleasant to differentiate directly, so we apply the same log trick as before. Taking the natural log turns the product into a sum and pulls each exponent down as a coefficient:

ln L(μ) = -n/2 · ln(2πσ²) - 1/(2σ²) · ∑(i=1 to n) (x_i - μ)²

The first term is a constant with respect to μ (it doesn't contain μ at all), so it vanishes upon differentiation. For the second term, differentiate the sum of squares with respect to μ: the derivative of (xᵢ − μ)² with respect to μ is −2(xᵢ − μ) by the chain rule (power rule on the square, times the derivative of the inner term with respect to μ, which is −1). So:

d/dμ [ln L(μ)] = -1/(2σ²) · ∑ [-2(x_i - μ)] = 1/σ² · ∑ (x_i - μ)

Setting this to zero (remember: σ² is a fixed positive known constant, so we can safely multiply both sides by it and it disappears from the equation):

∑(i=1 to n) (x_i - μ) = 0
∑ x_i - nμ = 0
μ̂ = (1/n) ∑ x_i = x̄

The maximum likelihood estimate of the population mean is exactly the sample mean, x̄ — the ordinary average you've been computing since Class 8. This is a genuinely important result, not a coincidence: it shows that a formula you already trusted on intuitive grounds (average the data to estimate the center) is also the answer a fully rigorous optimality principle demands. The second derivative here is −n/σ², which is negative for any σ² > 0 and any n ≥ 1, confirming this is indeed a maximum, not a minimum.

This also explains something you may have wondered about when you first met variance in CBSE statistics: why do we measure spread using squared deviations from the mean, Σ(xᵢ − x̄)², rather than some other measure? Part of the answer is exactly what you just derived — squared-deviation minimization (equivalently here, likelihood maximization under a Normal model) is what singles out the arithmetic mean as the "best" center point in the first place.

Verifying MLE Numerically

You don't need calculus to confirm an MLE result once you have candidate code — a grid search over possible values will locate the same peak, just less elegantly and less exactly. Here is a direct check for the IRCTC example:

import numpy as np
from math import comb

n, k = 20, 14
ps = np.linspace(0.01, 0.99, 99)
L = [comb(n, k) * p**k * (1 - p)**(n - k) for p in ps]
p_hat = ps[np.argmax(L)]
print(round(p_hat, 2))

Tracing this: ps holds 99 evenly spaced candidates from 0.01 to 0.99 in steps of 0.01 (so one of them lands almost exactly on 0.70). The list comprehension computes L(p) at every candidate using the same binomial formula we derived by hand. np.argmax(L) finds the index of the largest value in that list, and ps[...] reads off the corresponding p. Because we proved analytically that the true maximum sits at p = 0.70, and 0.70 is one of the 99 grid points tested, this program prints 0.7 — matching the calculus exactly. A grid search is a useful sanity check, but notice it only works because we tested enough points; the calculus derivation is what guarantees the exact answer for any dataset, not just ones where the true peak happens to land on a convenient grid point.

Where This Fits in Your Exam Preparation

MLE itself is not a named topic in the CBSE Class 10-12 core mathematics syllabus, so don't expect a board question titled "find the MLE." What it does directly strengthen is everything the boards and entrance exams build probability questions from: the binomial distribution, conditional probability, and Bayes' theorem, all of which are explicitly tested in IIT-JEE and BITSAT probability sections. Understanding why k/n is the "best" estimate — rather than just accepting it — gives you a sturdier grip on any question that asks you to reason about an unknown probability from observed frequency, including many olympiad-style combinatorics and probability problems that reward exactly this kind of first-principles reasoning over memorized formulas. If you continue toward data science or the newer GATE Data Science & AI (DA) paper, Maximum Likelihood Estimation appears explicitly in its probability and statistics syllabus — this chapter is the foundation that later course covers in full generality (multiple parameters, non-Normal distributions, numerical optimization when no closed-form solution exists).

Practice: Active Recall

1. Factory quality check. A Bengaluru electronics assembler tests 50 circuit boards from a new batch; 46 pass inspection. Using the binomial MLE result derived above, what is p̂, the maximum likelihood estimate of the true pass rate?
   Answer: p̂ = k/n = 46/50 = 0.92.
2. Why log first? Explain, in your own words, why we maximized ln L(p) instead of L(p) directly, and why this is mathematically legitimate.
   Answer: Taking the natural log turns a product into a sum (much easier to differentiate), and because ln(x) is strictly increasing, it never moves the location of the maximum — only its height. So the p that maximizes ln L(p) is guaranteed to be the same p that maximizes L(p) itself.
3. Numeric check. For n = 10 trials with k = 3 successes, compute L(0.2), L(0.3), and L(0.4) using L(p) = C(10,3)·p³·(1−p)⁷, and confirm which is largest. (C(10,3) = 120.)
   Hint: you should find L(0.3) is the largest of the three — consistent with p̂ = k/n = 0.3 exactly.
4. True or False, with justification. "Since L(0.7) ≈ 0.19 is bigger than L(0.5) ≈ 0.04, there is roughly a 19% chance that the true confirmation probability is 0.7."
   Answer: False. L(p) values are not probabilities of p; the likelihood curve doesn't integrate to 1 over p, and 0.19 only measures how probable the observed data would be under the assumption p = 0.7, not how probable p = 0.7 itself is.
5. Extend it. If a Normal-distribution MLE derivation (as done above) is repeated but now both μ and σ² are unknown, what extra step would the derivation need before you could solve for μ̂ and σ̂²?
   Hint: you would need to take partial derivatives of ln L with respect to both μ and σ² separately, and set both equal to zero simultaneously — a preview of multivariable optimization you'll formalize in Class 12 and beyond.

Summary

Maximum Likelihood Estimation answers a precise question: given data you've already observed, which value of an unknown parameter makes that exact data the most probable outcome? You build a likelihood function by writing the probability (or density) formula for your data and treating the parameter, not the data, as the variable. For n Bernoulli trials with k successes, maximizing the log-likelihood via calculus proves p̂ = k/n — turning an intuitive guess into a derived theorem, confirmed as a true maximum by a negative second derivative. For n Normal-distributed measurements with known variance, the same machinery proves the MLE of the mean is the ordinary sample mean, x̄. Throughout, remember the sharpest edge of the whole topic: a likelihood value is not a probability of the parameter — it measures how well a candidate parameter value explains data you already have, and MLE simply picks the candidate that explains it best.

Think About It

Think about this: How would you explain maximum likelihood estimation (mle) basics to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.