Bayesian Probability and Inference

A Positive Test Result — Should You Worry?

Imagine a public health screening camp sets up in your district to test for a certain condition — call it Disease X — that is known to affect roughly 1 in every 100 people in that population. The test used at the camp is genuinely good: if a person actually has Disease X, the test correctly flags them 95% of the time (this is called the test's sensitivity). If a person does not have Disease X, the test correctly clears them 90% of the time — which means it wrongly flags a healthy person as positive 10% of the time (this is the test's false positive rate).

You get tested. The result comes back positive. Before reading any further, make a guess: what do you think is the probability that you actually have Disease X?

Most people's gut answer is somewhere around 90% — after all, the test is "95% accurate," so a positive result should mean you're almost certainly sick. This gut answer is wrong, and it isn't wrong by a small margin. The real answer, which you will be able to compute exactly by the end of this chapter, is under 9%. The test result barely moves the needle. Understanding why a "95% accurate" test can still leave you more likely healthy than sick is exactly what Bayesian probability is for — and it is one of the most consequential ideas in all of applied mathematics, used everywhere from medical diagnosis to spam filters to spacecraft fault detection. By the end of this chapter you will not just know the answer to the puzzle above; you will be able to derive it from first principles and defend every step.

Refreshing Conditional Probability

Before we can build Bayes' theorem, we need one tool: conditional probability — the probability of an event given that we already know another event has happened. You've likely met the basic idea of probability as (favourable outcomes) / (total outcomes). Conditional probability is the same idea, except "total outcomes" shrinks to only the outcomes consistent with what you already know.

Here is a concrete anchor. Suppose your school has 200 students who tried out for the cricket team this year. Some of them also play in a district-level league outside school; some don't. Here is how they break down:

• Plays district league AND selected for school team: 24 students
• Plays district league but NOT selected: 16 students
• Does not play district league but IS selected: 30 students
• Does not play district league and NOT selected: 130 students

Total selected for the school team: 24 + 30 = 54 students, out of 200. So the plain probability that a randomly chosen student was selected is 54/200 = 0.27.

Now ask a sharper question: given that a student plays in the district league, what is the probability they were selected for the school team? We are no longer looking at all 200 students — we restrict our attention only to the 24 + 16 = 40 students who play district league, since that is the condition we're told is true. Of those 40, 24 were selected. So the conditional probability is 24/40 = 0.60 — much higher than the unconditional 0.27, which makes sense: playing competitive cricket outside school is good evidence of being good enough to make the school team.

We write this as P(Selected | League) = 0.60, read "the probability of Selected, given League." Formally, for any two events A and B in a sample space, with P(B) > 0:

P(A | B) = P(A and B) / P(B)

Check it against the cricket numbers: P(Selected and League) = 24/200 = 0.12, and P(League) = 40/200 = 0.20. So P(Selected | League) = 0.12 / 0.20 = 0.60. Matches exactly — restricting the sample space and dividing joint-by-marginal are the same operation.

Deriving Bayes' Theorem From Scratch

The conditional probability formula above can be rearranged into what's called the multiplication rule, just by multiplying both sides by P(B):

P(A and B) = P(A | B) x P(B)

Here is the key move. "A and B" is exactly the same event as "B and A" — set intersection doesn't care about order. So we can write the multiplication rule the other way around too, swapping the roles of A and B:

P(A and B) = P(B | A) x P(A)

Both expressions equal the same quantity, P(A and B), so we can set them equal to each other:

P(A | B) x P(B) = P(B | A) x P(A)

Divide both sides by P(B), and you have derived Bayes' theorem — not as a formula to memorise, but as a direct algebraic consequence of the definition of conditional probability applied twice:

P(A | B) = [ P(B | A) x P(A) ] / P(B)

That's the whole theorem. Its power comes from what it lets you do: it lets you flip a conditional probability around. Often you know P(B | A) easily (a scientist can directly measure "probability the test is positive, given the patient is sick") but you actually want P(A | B) (a doctor wants "probability the patient is sick, given the test is positive"). These two are not the same number, and Bayes' theorem is the precise, correct machine for converting one into the other.

In Bayesian language, each piece of this formula has a name, and learning this vocabulary is essential for exams and for reading further:

• Prior, P(A): what you believed about A before seeing any new evidence.
• Likelihood, P(B | A): how probable the evidence B is, assuming A is true.
• Posterior, P(A | B): your updated belief about A after incorporating evidence B. This is what you're solving for.
• Evidence (or marginal), P(B): the overall probability of seeing B at all, across every possible hypothesis.

That last term, P(B), needs its own tool when there are multiple competing hypotheses instead of just "A" and "not A." If H1, H2, ..., Hn are mutually exclusive events that cover every possibility (a partition of the sample space — exactly one of them must be true), then the total probability of any evidence E, regardless of which hypothesis is true, is the sum of each hypothesis's probability of producing that evidence, weighted by how likely that hypothesis was to begin with. This is the law of total probability:

P(E) = P(E|H1)P(H1) + P(E|H2)P(H2) + ... + P(E|Hn)P(Hn)

Substituting this into the denominator gives the general form of Bayes' theorem used for problems with several competing hypotheses (like our upcoming factory example):

P(Hi | E) = [ P(E|Hi) x P(Hi) ] / [ sum over all j of P(E|Hj) x P(Hj) ]

Solving the Health-Camp Puzzle

We now have every tool needed to solve the puzzle from the opening. Let D = "has Disease X" and let "+" = "tests positive." We were given:

• Prior: P(D) = 0.01, so P(not D) = 0.99
• Likelihood (sensitivity): P(+ | D) = 0.95
• False positive rate: P(+ | not D) = 0.10

First, apply the law of total probability to find P(+), the overall chance of a positive result across the whole population, sick or not:

P(+) = P(+|D)P(D) + P(+|not D)P(not D)
     = (0.95)(0.01) + (0.10)(0.99)
     = 0.0095 + 0.0990
     = 0.1085

Now apply Bayes' theorem:

P(D | +) = P(+|D) x P(D) / P(+)
         = 0.0095 / 0.1085
         = 0.0876...
         ≈ 8.8%

So a positive result on a "95% accurate" test leaves you with only an 8.8% chance of actually having Disease X — nowhere near the 90%+ that gut intuition suggests. The tree diagram below shows exactly where this number comes from: the population splits first by the prior (99% healthy, only 1% sick), and only then does the test's accuracy act on each branch. Because healthy people vastly outnumber sick people, even a small 10% false-positive rate among the huge healthy group (0.10 x 0.99 = 0.0990) produces far more false alarms in absolute terms than the true positives coming from the tiny sick group (0.95 x 0.01 = 0.0095). This is called the base rate effect: when a condition is rare, false positives from the enormous healthy majority swamp true positives from the tiny affected minority, even when the test itself is fairly accurate.

A Common and Dangerous Misconception: The Prosecutor's Fallacy

The single most common error in this topic — common enough to have its own name, the prosecutor's fallacy — is treating P(evidence | hypothesis) as if it were the same number as P(hypothesis | evidence). In our example, that would mean reading "the test is positive in 95% of people who have the disease" as if it said "95% of people who test positive have the disease." These are two completely different conditional probabilities, computed with two different denominators, and mixing them up is exactly the mistake that produced the wrong 90% gut guess at the start of this chapter.

The name comes from courtrooms: a prosecutor might argue "the probability of this DNA match occurring by pure chance is 1 in a million, therefore there is only a 1-in-a-million chance the defendant is innocent." The first part of that sentence is a likelihood, P(match | innocent) — a statement about how rare the evidence is if you assume innocence. The second part quietly swaps it for a posterior, P(innocent | match), without ever bringing in the prior probability that a random person in the city is even a plausible suspect, or applying Bayes' theorem correctly. If the city has ten million people, roughly ten of them would match by chance alone — a defendant matching is nowhere near proof of guilt on that evidence alone. Whenever you see a probability statement in a word problem, always pin down precisely which variable is on the left of the "|" and which is on the right — they are not interchangeable, and no amount of the number being large or small changes that.

Updating Again: Sequential Evidence

One of the deepest ideas in Bayesian inference is that it doesn't stop after one piece of evidence. Your posterior after the first test becomes your prior for interpreting a second, independent test. This is exactly what a careful doctor does in practice: order a confirmatory second test rather than trusting one result blindly.

Suppose, after that first positive result, you take a second, independent test with the same accuracy characteristics. Your new prior is the old posterior, 0.0876. Apply the same machinery again:

P(++) = P(+|D)(0.0876) + P(+|not D)(1 - 0.0876)
      = (0.95)(0.0876) + (0.10)(0.9124)
      = 0.08322 + 0.09124
      = 0.17446

P(D | + ,+) = 0.08322 / 0.17446 ≈ 0.477 ≈ 47.7%

Two independent positive results push your probability of actually having Disease X from 8.8% up to roughly 47.7% — a huge jump, but still not certainty. Run the same update a third time using 0.477 as the new prior, and the posterior climbs to approximately 89.7%. Notice the pattern: each additional independent positive test multiplies your evidence, and the posterior climbs steeply once it clears the base-rate handicap. This is precisely why repeated or confirmatory testing is standard medical and scientific practice — a single positive result on a rare condition is weak evidence on its own, but a short sequence of independent positive results compounds into strong evidence very quickly.

A Second Worked Example: The Three-Machine Factory Problem

You'll meet this exact question template again and again in Class 12 boards and in entrance exams like JEE Main and BITSAT, so it's worth building real fluency with it now, while the underlying logic is still fresh rather than memorised as a formula. A factory has three machines producing the same component:

• Machine M1 produces 30% of all output, with a 2% defect rate.
• Machine M2 produces 45% of all output, with a 1% defect rate.
• Machine M3 produces 25% of all output, with a 4% defect rate.

A component is picked at random from the factory's total output and found to be defective. What is the probability it came from each machine? Here {M1, M2, M3} form a partition — every component came from exactly one machine — so this calls for the general form of Bayes' theorem with the law of total probability in the denominator. Let D = "component is defective." First, find P(D) by summing each machine's contribution — its output share times its own defect rate:

P(D) = P(D|M1)P(M1) + P(D|M2)P(M2) + P(D|M3)P(M3)
     = (0.02)(0.30) + (0.01)(0.45) + (0.04)(0.25)
     = 0.0060 + 0.0045 + 0.0100
     = 0.0205

Now apply Bayes' theorem to each machine in turn, dividing its own contribution by the total:

P(M1|D) = 0.0060 / 0.0205 ≈ 0.293  (29.3%)
P(M2|D) = 0.0045 / 0.0205 ≈ 0.220  (22.0%)
P(M3|D) = 0.0100 / 0.0205 ≈ 0.488  (48.8%)

These three posteriors should sum to almost exactly 1.0 (they sum to 100.1% here purely because each was rounded to three significant figures before adding — always expect a rounding residue of a tenth of a percent or so when you check your work this way, rather than treating a small mismatch as a sign of an arithmetic error).

Now look closely at what the numbers actually say, because the result is genuinely counterintuitive. M2 produces the largest share of the factory's total output — 45%, nearly half of everything made — yet it ends up the least likely source of a randomly selected defective part, at only 22.0%. Meanwhile M3, which produces the smallest share of output (just 25%), turns out to be the most likely source, at 48.8%. The explanation is the same base-rate logic from the disease example, just running in the opposite direction: M2's defect rate (1%) is disproportionately low relative to its huge output, so despite making almost half the parts, it contributes only a small slice of the defects. M3's defect rate (4%) is disproportionately high relative to its modest output share, so it punches well above its weight in the defect count. Output share alone does not tell you the likely culprit — you must weight it by the failure rate, and Bayes' theorem is precisely the tool that does that weighting correctly.

Where Bayesian Reasoning Reappears

The same three-step pattern — start with a prior, multiply by a likelihood for each hypothesis, normalise by the total — is the engine behind a family of tools called naive Bayes classifiers, used in spam filters that estimate P(spam | these words) from the prior rate of spam email and the likelihood of each word appearing in spam versus genuine mail. It's also the formal basis for how diagnostic support software combines a patient's prior risk factors with new test results, and how fault-detection systems on spacecraft and industrial plants combine sensor readings with known prior failure rates of individual components — structurally the exact factory problem above, just with sensors standing in for machines. The mathematics you derived in this chapter from a single line, P(A|B) = P(A and B)/P(B), is the same mathematics running underneath all of these.

Check Your Understanding

Work through these before checking the answers beneath each one — the goal is to reconstruct the reasoning, not just recall a number.

1. Q1. An airport security scanner correctly flags a prohibited item 99% of the time when one is present, and correctly clears an innocent bag 97% of the time (so it falsely flags an innocent bag 3% of the time). Only 1 in 10,000 bags actually contains a prohibited item. A bag is flagged. What is the probability it actually contains a prohibited item?
   Answer: P(item)=0.0001, P(flag|item)=0.99, P(flag|no item)=0.03. P(flag) = (0.99)(0.0001) + (0.03)(0.9999) = 0.000099 + 0.029997 = 0.030096. P(item|flag) = 0.000099/0.030096 ≈ 0.0033, about 0.33%. Despite a flag, the bag is almost certainly innocent — the base rate of prohibited items is simply too low for a 3% false-positive rate not to dominate.
2. Q2. In the three-machine factory problem, suppose instead a component is picked at random and found to be non-defective. Without recomputing from scratch, explain in one sentence why M2 (the machine with the lowest defect rate) should now have the highest posterior probability of the three, and roughly why.
   Answer: Because M2 makes both the largest share of total output and, among the good parts specifically, contributes an even larger share (since so little of its output is defective) — a randomly chosen good part is disproportionately likely to have come from the machine that rarely fails and also makes almost half of everything.
3. Q3. Explain, in your own words, why "the test is 95% accurate" is not by itself enough information to answer "given a positive result, what's the probability I'm sick?" Name the missing ingredient.
   Answer: The missing ingredient is the prior, P(disease) — the base rate of the condition in the population. Accuracy alone describes the likelihood, P(positive|disease), but the posterior also depends on how rare or common the disease is to begin with; the same test accuracy produces wildly different posteriors for a rare disease versus a common one.

Summary

Conditional probability, P(A|B) = P(A and B)/P(B), restricts your sample space to only the outcomes consistent with what you already know. Applying this definition twice — once for P(A|B) and once for P(B|A), both describing the same joint event — and equating the results gives Bayes' theorem: P(A|B) = P(B|A)P(A)/P(B). When there are several competing hypotheses, the law of total probability builds the denominator by summing each hypothesis's prior weighted by its own likelihood of producing the observed evidence. The vocabulary — prior, likelihood, posterior, evidence — describes the same update procedure every time: start with what you believed, weigh it by how well it explains what you just observed, and normalise. The central lesson worth carrying forward is that a highly accurate test or a low probability of coincidence is not, by itself, the answer to "given this evidence, what's the probability of the hypothesis" — the prior always matters, sometimes enough to completely overturn what instinct suggests, and posteriors compound cleanly across repeated independent evidence, which is exactly why confirmatory testing works.