Bootstrapping: Confidence Without Theory

A CI that goes below the smallest number you measured

Eight students at your school attempt the same JEE-level definite-integral problem and you time how long each one takes, in minutes: 9, 10, 11, 12, 13, 14, 15, 40. Seven of them cluster tightly between 9 and 15 minutes; one student gets stuck, backtracks twice, and finally finishes in 40. That single 40 is real data, not a typo — some students genuinely go down a wrong substitution and claw their way back.

You want a 90% confidence interval for the true mean solving time across all students who might attempt this problem. The standard Class 11 recipe is: compute the sample mean and sample standard deviation, then use the t-distribution because the population standard deviation is unknown and n is small. Let's actually run it, carefully, because the answer is going to teach us something the formula itself can't tell you.

Sample mean: x̄ = (9 + 10 + 11 + 12 + 13 + 14 + 15 + 40) / 8 = 124 / 8 = 15.5 minutes.

Sample variance uses squared deviations from 15.5: (−6.5)² = 42.25, (−5.5)² = 30.25, (−4.5)² = 20.25, (−3.5)² = 12.25, (−2.5)² = 6.25, (−1.5)² = 2.25, (−0.5)² = 0.25, and (24.5)² = 600.25. These sum to 714.0. Dividing by n − 1 = 7 gives sample variance s² = 102.0, so s = √102.0 ≈ 10.10.

Standard error: SE = s / √n = 10.10 / √8 = 10.10 / 2.828 ≈ 3.57.

For a 90% CI with df = n − 1 = 7, the two-tailed critical value is t(0.05, 7 df) = 1.895 (a standard table value). Margin of error = 1.895 × 3.57 ≈ 6.77.

So the classical interval is 15.5 ± 6.77, i.e. (8.73, 22.27) minutes.

Look closely at that lower bound: 8.73. Every single student in your sample took at least 9 minutes. The formula is confidently telling you the true average could be as low as 8.73 — a value nobody in your data ever recorded, and one that isn't even physically plausible given how the rest of the sample behaves. This isn't a calculation error; it's a symptom. The t-interval is built on an assumption — that the underlying population is close to normally distributed, or that n is large enough for the Central Limit Theorem to smooth things out — and with n = 8 and one dominating outlier, neither condition holds comfortably. The formula doesn't know your data is skewed. It just multiplies and reports a number.

This chapter is about what to do when you don't want to lean on that assumption at all — when you want a confidence interval built directly from the data you actually have, with no claim about the shape of the population behind it. That technique is called bootstrapping, formalized by the statistician Bradley Efron in 1979.

The idea: treat your sample as a stand-in population

Here is the logical snag that classical confidence intervals live inside. To know how much a sample mean varies from sample to sample, you would ideally draw many independent samples from the population and watch the means bounce around. But you don't have the population — you have exactly one sample of size 8. Classical statistics escapes this by assuming a distributional shape (normal) and deriving, through probability theory, exactly how the sample mean's variability behaves under that assumption. That's powerful when the assumption is reasonable, and misleading when it isn't.

Bootstrapping escapes the same snag differently. It uses the plug-in principle: since you don't know the true population distribution, use the best estimate of it that you actually possess — the empirical distribution, which places probability 1/n on each of your n observed data points and zero probability everywhere else. Then, instead of sampling from the unknown population (impossible), you sample from your own known data (trivial on a computer): you draw n values from your dataset, with replacement, treating your 8 numbers as if they were the entire universe of possible outcomes.

"With replacement" is the detail that makes this work. If you sampled 8 values from your 8 data points without replacement, you'd get the exact same 8 numbers back every time, in some order — no new information, no variability. Sampling with replacement means every draw independently picks any of the 8 original values, so some values get picked two or three times in a resample and others get skipped entirely. That's what injects variability — the same kind of variability that would exist if you had actually gone out and collected a fresh sample of 8 students.

The algorithm, precisely

Given original data of size n and a statistic of interest T (mean, median, standard deviation — anything you can compute from a sample):

1. Draw n values from the original data independently and uniformly at random, with replacement. Call this a bootstrap resample.
2. Compute the statistic T on this resample. Call the result a bootstrap replicate.
3. Repeat steps 1–2 a large number of times, B (typically 1,000 to 10,000), producing B bootstrap replicates.
4. The collection of all B replicates is the bootstrap distribution of T — your simulated stand-in for "what would happen if I could collect many real samples."

Every step here uses only arithmetic and a random-number generator. There is no assumption anywhere in this list about the population being normal, symmetric, or any particular shape.

Doing it by hand: five resamples from eight numbers

A computer will eventually do this thousands of times, but doing it by hand a few times first is the only way to actually feel what's happening. Number the original data by position: x1 = 9, x2 = 10, x3 = 11, x4 = 12, x5 = 13, x6 = 14, x7 = 15, x8 = 40.

Each resample below was formed by picking 8 index numbers from 1–8 with repetition allowed (you can do this yourself with a random-integer generator, or by drawing chits with replacement from a bag numbered 1–8):

• Resample 1 — indices (1,1,3,4,4,6,7,8) → values 9,9,11,12,12,14,15,40. Sum = 122, mean = 122/8 = 15.25.
• Resample 2 — indices (2,3,3,5,5,5,7,8) → values 10,11,11,13,13,13,15,40. Sum = 126, mean = 126/8 = 15.75.
• Resample 3 — indices (1,2,4,5,6,6,7,7) → values 9,10,12,13,14,14,15,15. Sum = 102, mean = 102/8 = 12.75. Notice: the 40 wasn't drawn at all in this resample, and the mean drops sharply.
• Resample 4 — indices (8,8,1,2,3,4,5,6) → values 40,40,9,10,11,12,13,14. Sum = 149, mean = 149/8 = 18.625. Here 40 got drawn twice, and the mean jumps well above the original 15.5.
• Resample 5 — indices (3,4,5,6,7,1,2,3) → values 11,12,13,14,15,9,10,11. Sum = 95, mean = 95/8 = 11.875.

Five resamples, five different means: 15.25, 15.75, 12.75, 18.625, 11.875. Compare that spread to the original single mean of 15.5 — you can already see the outline of a distribution forming, and you can see exactly why it has that shape: whenever the resample happens to skip the 40, the mean sits in the low-to-mid 12s; whenever it grabs the 40 once, the mean sits near 15–16; on the rare resample that grabs 40 twice, the mean jumps toward 19 or higher. The bootstrap distribution is going to be right-skewed for a completely traceable, mechanical reason — not because of any assumption, but because of exactly how often the outlier gets picked.

How many resamples are even possible?

This is worth pausing on, because it's a clean application of the multiplication principle you use in permutations-and-combinations problems. Each of the 8 positions in a resample is filled independently by choosing among the 8 original values, and repeats are allowed. So the number of distinct ordered bootstrap resamples is 8 × 8 × 8 × 8 × 8 × 8 × 8 × 8 = 8⁸ = 16,777,216. In general, for a sample of size n, there are nⁿ possible ordered resamples. This is exactly the "arrangements with repetition allowed" count from the fundamental principle of counting — the same logic that tells you there are 10⁴ four-digit PINs. It's also why, in practice, nobody tries to list every possible resample: even for a modest n = 8, there are over 16 million of them, so instead we draw a large random subset of size B (say 10,000) and treat that as a good approximation of the full bootstrap distribution.

From resamples to an interval: the percentile method

Once you have B bootstrap replicates, sort them from smallest to largest. A (1 − α) bootstrap confidence interval, by the percentile method, is simply the interval between the (α/2)-th percentile and the (1 − α/2)-th percentile of that sorted list. For a 90% CI, α = 0.10, so that means reading off the 5th and 95th percentiles of your (say) 10,000 bootstrap means. There is no critical-value table, no assumption of symmetry, and no formula for the standard error required. The data generates its own interval.

Running it for real: code

Doing thousands of resamples by hand is not the point — a computer does steps 1–3 of the algorithm in a fraction of a second. Here is the whole procedure in Python, for both the mean and, just as easily, the median:

import numpy as np

data = np.array([9, 10, 11, 12, 13, 14, 15, 40])
n = len(data)
B = 10000  # number of bootstrap resamples

rng = np.random.default_rng(seed=42)
boot_means = np.empty(B)
boot_medians = np.empty(B)

for i in range(B):
    resample = rng.choice(data, size=n, replace=True)
    boot_means[i] = resample.mean()
    boot_medians[i] = np.median(resample)

ci_mean = np.percentile(boot_means, [5, 95])
ci_median = np.percentile(boot_medians, [5, 95])

print("Original mean:", data.mean())
print("90% bootstrap CI for the mean:", ci_mean)
print("Bootstrap SE of the mean:", boot_means.std(ddof=1))
print("Original median:", np.median(data))
print("90% bootstrap CI for the median:", ci_median)

Trace what happens: the loop runs 10,000 times; each pass calls rng.choice(data, size=8, replace=True), which is exactly step 1 of the algorithm — 8 draws from the 8-element array, with replacement, each draw independent — then records the resample's mean and median into two arrays. np.percentile(boot_means, [5, 95]) reads off the 5th and 95th percentile of those 10,000 numbers, which is exactly the percentile-method interval defined above. The exact printed decimals depend on the random-number generator's internal state, but the qualitative behaviour is guaranteed by the mechanics we traced by hand: the probability a given resample contains zero copies of the outlier is (7/8)⁸ ≈ 0.344, so roughly a third of the 10,000 resamples look like Resample 3 above (mean well below 15.5), while the rest include the 40 at least once and get pulled upward — producing a bootstrap CI that is right-skewed and fully contained inside [9, 40], unlike the classical interval that dipped below 9.

The bootstrap standard error, and why the trick isn't cheating

You can also just take the standard deviation of the B bootstrap replicates themselves — that's boot_means.std(ddof=1) in the code above — and call it the bootstrap standard error. As B grows large, this converges to the standard deviation of the sampling distribution implied by treating your data as the truth, and — this is the theoretical result Efron proved — that quantity converges to the true standard error as n grows, under quite general conditions on the population, no normality required. The formal justification rests on the Glivenko–Cantelli theorem, which guarantees that the empirical distribution built from your sample converges uniformly to the true population distribution as n grows. In plain terms: your sample, especially a reasonably sized one, really is a decent miniature of the population, so resampling from it behaves like resampling from the population itself.

That word "especially" matters. Bootstrapping is not magic — it cannot manufacture information that was never collected. If your original sample of 8 happened to badly misrepresent the population (say, by pure chance, you'd sampled unusually fast solvers only), every bootstrap resample would inherit that same bias, because every resample is drawn only from those same 8 numbers. The bootstrap distribution correctly reflects the uncertainty coming from having only n = 8 data points; it cannot fix a sample that isn't representative to begin with.

A confidence interval the CBSE formula sheet has no entry for: the median

Here is where bootstrapping earns its keep beyond "an alternative to the t-interval." Ask for a 90% confidence interval for the population median solving time, instead of the mean. Try to recall a closed-form formula for the standard error of a sample median from your Class 11–12 statistics chapter — there isn't a simple one on your formula sheet, because the sampling distribution of the median doesn't have a clean algebraic form the way the mean's does under the Central Limit Theorem. Classical statistics handles this case with more advanced theory (order statistics, asymptotic normality of the median) that goes well beyond school level.

The bootstrap code above handles it with one extra line: np.median(resample) instead of resample.mean(). Nothing else about the algorithm changes — draw with replacement, compute the statistic, repeat, take the 5th/95th percentiles. This is the real power of bootstrapping: it turns "derive a new formula for every new statistic" into "write one extra line of code." The same recipe, unchanged, gives you a CI for a median, a trimmed mean, a correlation coefficient, a ratio of two means, or almost anything else you can compute from a sample.

Two things students get wrong

Misconception 1: "More bootstrap resamples (B) means a more accurate estimate of the true population parameter." This confuses two different sources of error. Increasing B from 1,000 to 100,000 only reduces simulation noise — it makes your estimate of the bootstrap distribution itself smoother and more stable, the same way rolling a die more times gives you a cleaner estimate of its true probabilities. It does nothing to shrink the interval's width, because the width is fundamentally set by n = 8, the size of your original sample. Run B = 10,000 or B = 10,000,000 on the same 8 numbers and you'll get almost the same CI either way, because both are approximating the same underlying bootstrap distribution, which is fixed the moment your 8 data points are fixed. To get a genuinely narrower interval you need more original data, not more resamples of the same data.

Misconception 2: "A bootstrap resample should be smaller than the original sample, since you're 'reusing' data." Each resample must be the same size n as the original sample — 8 draws from 8 values here, not 4 or 5. Drawing fewer than n values with replacement gives a statistic with a different (usually larger) variance than the one you're trying to estimate, so the resulting interval would be systematically wrong. "With replacement" is what supplies the variability; the sample size stays fixed at n throughout.

Where this sits relative to CBSE, JEE, and beyond

Bootstrapping itself is not a CBSE board-exam topic, and you won't be asked to compute one in a JEE Main paper. What you will be tested on, repeatedly, are its exact building blocks: sample mean and standard deviation (Class 11, Statistics), the multiplication principle for counting arrangements with repetition (Class 11, Permutations and Combinations — precisely the nⁿ count above), percentiles and quartiles, and the logic of confidence and probability (Class 12). Entrance exams for statistics-heavy programs (ISI, CMI) and Olympiad-adjacent combinatorics problems draw on exactly this kind of "count the arrangements, then reason about the distribution" thinking. Treat this chapter as the place those separately taught pieces — combinatorics, mean and variance, percentiles — visibly assemble into a single, genuinely modern statistical method, one that any working data scientist reaches for before reaching for a formula sheet.

Check your understanding

• For the 8-number dataset in this chapter, what is the exact probability that a single bootstrap resample of size 8 consists of the value 40 appearing all 8 times? (Hint: each of the 8 draws is independent and uniform over 8 values.)
• Explain, using the idea of the empirical distribution, why a bootstrap resample is always drawn "with replacement" and never "without replacement."
• A classmate runs the bootstrap code above but sets size=4 inside rng.choice instead of size=n. Explain concretely why the resulting confidence interval would be untrustworthy, even though the code runs without any error.
• Two students both bootstrap the same 8-number dataset: Student A uses B = 500 resamples, Student B uses B = 50,000. Their two 90% CIs come out nearly identical in width. Is this a bug, or expected? Justify using the distinction between n and B.
• Why does a closed-form formula like x̄ ± t·(s/√n) exist for the mean but not for the median, and how does that gap specifically motivate using bootstrapping for the median?

Summary

When a sample is small, skewed, or built around a statistic with no tidy formula (like the median), classical confidence intervals either force an unjustified normality assumption or simply don't exist in closed form. Bootstrapping sidesteps both problems by treating the observed sample as a stand-in population and resampling from it, with replacement, thousands of times — computing the statistic of interest on each resample and reading the confidence interval directly off the resulting distribution's percentiles. The technique costs nothing in extra assumptions, generalizes to almost any statistic with one extra line of code, and — as the outlier-laden solving-time example showed — can visibly outperform the textbook formula precisely in the situations where that formula is shakiest. What it cannot do is manufacture certainty your original sample never earned: more resamples smooth the simulation, but only more real data narrows the interval.

Think About It

Think about this: How would you explain bootstrapping: confidence without theory to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.