Bayesian Inference: Updating Beliefs with Evidence

A health camp at a school in a small town screens 10,000 students for a rare infection using a rapid test. The test is 99% accurate at catching the infection when it is present, and it correctly clears 95% of healthy students. Your result comes back positive. A friend, seeing the "99% accurate" line on the pamphlet, is convinced you must have the infection. You do the arithmetic properly and discover something that surprises almost everyone the first time: the chance you actually have the infection, given this positive result, is only about 17%. Not 99%. Not even close to it.

This is not a trick question and it is not a flaw in the test. It is a direct, unavoidable consequence of how probability works when an event is rare. Understanding exactly why requires a tool that is arguably the single most important idea in statistics and machine learning: Bayesian inference — the disciplined process of updating a probability (a "belief") in light of new evidence, using the mathematics of conditional probability rather than intuition. Spam filters, medical diagnosis systems, spacecraft navigation software, and recommendation engines all run on a version of the calculation you are about to learn.

Why Intuition Fails: Counting People, Not Percentages

The cleanest way to see what went wrong with your friend's reasoning is to stop thinking in percentages and start counting actual people. Suppose in this population of 10,000 students, the infection genuinely affects 1% of them — that is the prior information, what we believe before any test is run.

• Infected: 10,000 × 0.01 = 100 students.
• Not infected: 10,000 − 100 = 9,900 students.

Now apply the test to each group separately.

Among the 100 infected students, the test has 99% sensitivity (it correctly flags true cases), so:

• True positives: 100 × 0.99 = 99 students test positive.
• False negatives: 100 × 0.01 = 1 student tests negative despite being infected.

Among the 9,900 healthy students, the test has 95% specificity (it correctly clears healthy people), which means it wrongly flags the other 5%:

• False positives: 9,900 × 0.05 = 495 students test positive despite being healthy.
• True negatives: 9,900 × 0.95 = 9,405 students correctly test negative.

Now count everyone who tested positive, infected or not: 99 + 495 = 594 students. Of those 594 positive results, only 99 are true infections. So:

P(infected | tested positive) = 99 / 594 ≈ 0.1667 = 16.7%

The 99%-accurate test still produces far more false alarms than true alarms among the positives, because there are simply so many more healthy students to draw false positives from than infected students to draw true positives from. This is called the base-rate effect: when the underlying event is rare, even a very accurate test generates a positive-result pool dominated by false alarms. Below is the same reasoning as a tree, so you can see the four outcome groups and how the answer is built purely from the two "positive" branches.

Formalizing It: Deriving Bayes' Theorem

The tree diagram works, but it is slow to redraw every time. We need a formula. It falls straight out of the definition of conditional probability, which you have already met: for two events A and B with P(B) > 0,

P(A | B) = P(A ∩ B) / P(B)

This just says "the probability of A, restricted to the world where B happened, is the fraction of B's outcomes that also satisfy A." Now write the same definition with the roles of A and B swapped:

P(B | A) = P(A ∩ B) / P(A)  ⟹  P(A ∩ B) = P(B | A) · P(A)

Both expressions describe the same intersection P(A ∩ B), so substitute the second into the first:

P(A | B) = [P(B | A) · P(A)] / P(B)

That is Bayes' theorem, and nothing about it is mysterious — it is conditional probability applied twice and rearranged. In inference, we relabel A as a hypothesis H (e.g. "has the infection") and B as observed evidence E (e.g. "tested positive"), giving the form you will use constantly:

P(H | E) = P(E | H) · P(H) / P(E)

Each term has a name, and learning these names is not decoration — it is what lets you set up any Bayesian problem correctly:

• P(H) — the prior: what you believed about H before seeing this evidence (here, the 1% base rate).
• P(E | H) — the likelihood: how probable the evidence is if H is true (here, the 99% sensitivity).
• P(H | E) — the posterior: your updated belief about H after seeing the evidence. This is what you are solving for.
• P(E) — the evidence (or marginal likelihood): the overall probability of seeing E at all, across every way it could have happened. It acts as a normalizing constant.

P(E) is computed with the law of total probability, splitting evidence across the hypothesis being true or false:

P(E) = P(E | H) · P(H) + P(E | ¬H) · P(¬H)

Plugging in the health-camp numbers — P(H) = 0.01, P(E|H) = 0.99 (sensitivity), P(E|¬H) = 1 − 0.95 = 0.05 (false-positive rate) — reproduces exactly the tree-diagram answer:

P(E)   = 0.99(0.01) + 0.05(0.99) = 0.0099 + 0.0495 = 0.0594
P(H|E) = 0.0099 / 0.0594 ≈ 0.1667

Here is that computation as code, so you can trace exactly how each named term maps onto a line:

def bayes_posterior(prior, sensitivity, specificity):
    p_e_given_h = sensitivity              # P(E | H)
    p_e_given_not_h = 1 - specificity      # P(E | not H)
    p_e = p_e_given_h * prior + p_e_given_not_h * (1 - prior)  # P(E)
    return (p_e_given_h * prior) / p_e     # P(H | E)

posterior = bayes_posterior(0.01, 0.99, 0.95)
print(round(posterior, 4))
# Trace: p_e_given_h = 0.99, p_e_given_not_h = 0.05
#        p_e = 0.99*0.01 + 0.05*0.99 = 0.0099 + 0.0495 = 0.0594
#        return 0.0099 / 0.0594 = 0.16666...
# Output: 0.1667

This is the standard shape of every Bayesian inference problem you will meet in Class 12 Probability, JEE, KVPY, or any introductory machine-learning course: identify the prior, identify the likelihood of the evidence under each hypothesis, and combine them through Bayes' theorem to get the posterior.

A Second Worked Example: When a "95% Accurate" Fraud Detector Isn't

Suppose a bank builds a machine-learning model that flags suspicious UPI transactions. Fraudulent transactions are rare — say, hypothetically, 1 in 1,000 transactions (P(H) = 0.001). The model catches 95% of actual fraud when it occurs (sensitivity), but also mistakenly flags 2% of completely legitimate transactions (false-positive rate). If a transaction gets flagged, what is the probability it is actually fraudulent?

P(E)   = 0.95(0.001) + 0.02(0.999) = 0.00095 + 0.01998 = 0.02093
P(H|E) = 0.00095 / 0.02093 ≈ 0.0454 = 4.5%

Even with a "95% accurate" model, a single flag means the transaction is fraudulent only about 4.5% of the time — because legitimate transactions vastly outnumber fraudulent ones, so a 2% error rate on the huge legitimate pool generates more false alarms than a 95% catch rate on the tiny fraud pool. This is exactly why real fraud systems do not auto-block on one flag; they route it for additional evidence, which is where Bayesian inference becomes genuinely iterative.

Sequential Updating: Today's Posterior Is Tomorrow's Prior

The word "inference" in "Bayesian inference" signals something the health-camp example only hinted at: Bayes' theorem is not a one-shot calculation. Once you compute a posterior from one piece of evidence, that posterior becomes the prior for the next piece of evidence. Beliefs accumulate as data arrives — this is the core Bayesian idea, and it is what powers real classifiers.

The bookkeeping is cleanest in odds form. Define the odds of H as O(H) = P(H) / P(¬H). Writing Bayes' theorem for H and for ¬H and dividing one by the other, the P(E) terms cancel:

O(H | E) = [P(E|H) / P(E|¬H)] · O(H)
         =        likelihood ratio · prior odds

The quantity P(E|H) / P(E|¬H) is called the likelihood ratio: how much more (or less) likely this evidence is under H than under ¬H. Posterior odds are just prior odds scaled by that ratio — and when two pieces of evidence are independent given H, their likelihood ratios simply multiply.

Return to the fraud example. Prior odds: 0.001/0.999 ≈ 0.001001. The transaction-flag likelihood ratio is 0.95/0.02 = 47.5, giving posterior odds 0.001001 × 47.5 ≈ 0.0475, i.e. posterior probability ≈ 4.5% — matching the direct calculation above, as it must.

Now suppose a second, independent signal also fires: the transaction comes from a device fingerprint never seen on this account before. Say P(mismatch | fraud) = 0.6 and P(mismatch | legitimate) = 0.03, giving a likelihood ratio of 0.6/0.03 = 20. Multiply it into the running odds:

posterior odds after both signals = 0.001001 × 47.5 × 20 ≈ 0.951
posterior probability = 0.951 / (1 + 0.951) ≈ 0.487 = 48.7%

One flag moved belief from 0.1% to 4.5%. A second, independent flag moved it from 4.5% all the way to 48.7% — evidence compounds multiplicatively in odds space, which is why stacking several weak, independent signals can be far more powerful than trusting one strong one. This multiply-the-likelihoods-and-assume-independence trick, applied across many features at once (words in an email, metadata in a transaction), is precisely the mechanism behind the Naive Bayes classifier, one of the oldest working spam filters in machine learning — "naive" because it assumes the evidence features are conditionally independent given the class, which is rarely exactly true but works remarkably well in practice.

The Misconception That Trips Up Even Careful Students

The single most common error in this entire topic is confusing P(E | H) with P(H | E) — treating "the test is positive 99% of the time when you're infected" as if it meant "you're infected 99% of the time when the test is positive." These are different conditional probabilities, and as the health-camp example shows, they can differ by a factor of six. This confusion has a name — the transposed conditional (sometimes called the prosecutor's fallacy in legal contexts) — and it appears constantly in casual reasoning about medical tests, forensic evidence, and news statistics. Consider a hypothetical forensic scenario: suppose a particular physical trait matches only 1 person in 10,000 at random (P(match | innocent) = 0.0001), and a suspect's trait matches. It is tempting to say "there's a 0.9999 probability of guilt." But that ignores the prior — how many people could plausibly have committed the act in the first place — exactly the way ignoring the 1% base rate wrecked the test-accuracy intuition. Without a sensible prior, P(E | H) alone tells you nothing about P(H | E).

Bayesian vs. Frequentist Thinking

It is worth naming the philosophical shift, because it is what makes this "inference" and not just "a formula." Classical (frequentist) statistics treats a hypothesis as fixed but unknown, and asks how likely the observed data would be under that fixed hypothesis. Bayesian inference instead treats the hypothesis itself as having a probability — a degree of belief — that gets explicitly revised as data arrives, starting from a stated prior. This is why Bayesian methods are so natural for machine learning: a model's confidence about a label, a parameter, or a prediction is exactly a belief that should shift as more training examples (evidence) come in, and Bayes' theorem is the mathematically consistent way to do that shifting instead of updating beliefs by gut feeling.

Practice: Test Your Understanding

1. An ISRO ground-station anomaly detector flags unusual telemetry. Anomalies genuinely occur in 0.5% of readings. The detector has 90% sensitivity and a 4% false-positive rate. Given a flag, what is P(true anomaly | flagged)? (Answer: P(E) = 0.90×0.005 + 0.04×0.995 = 0.0045+0.0398=0.0443; posterior = 0.0045/0.0443 ≈ 10.2%.)
2. A bag contains 3 fair coins and 1 two-headed coin. You pick one at random and flip it twice, getting heads both times. What is the probability you picked the two-headed coin? (Prior for two-headed = 1/4. P(HH | fair)=1/4, P(HH | two-headed)=1. P(HH)=1/4×1/4 + 1×3/4×... careful: P(two-headed)=1/4, P(fair)=3/4. P(HH)=1×(1/4)+ (1/4)×(3/4) = 0.25+0.1875=0.4375. Posterior = 0.25/0.4375 ≈ 57.1%.)
3. Explain, in your own words, why a highly accurate test can still produce mostly false positives when the condition it detects is rare. Use the term "base rate" in your answer.
4. Two independent lab tests are run on the same sample, with likelihood ratios 15 and 8 respectively, on a hypothesis with prior odds 0.002. Find the posterior probability after both tests. (Posterior odds = 0.002 × 15 × 8 = 0.24; posterior probability = 0.24/1.24 ≈ 19.4%.)

Summary

Bayesian inference is the rule-governed process of turning a prior belief P(H) into a posterior belief P(H | E) once evidence E arrives, via P(H | E) = P(E | H)·P(H) / P(E). The result depends critically on the prior — rare hypotheses stay improbable even after seemingly strong evidence, which is the base-rate effect behind the health-camp paradox and the fraud-detector example. The odds form turns repeated evidence into simple multiplication of likelihood ratios, which is both the practical way to do sequential updating and the mechanism inside classifiers like Naive Bayes. The one error to permanently avoid is swapping P(E | H) for P(H | E) — they are not the same number, and mistaking one for the other is the most common way this topic goes wrong, in a classroom, in a courtroom, or in a machine-learning pipeline.

Think About It

Think about this: How would you explain bayesian inference: updating beliefs with evidence to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.