Confidence Intervals — What '95%' Really Means
What '95% confident' actually means
This concept builds on Normal Distribution — The Bell Curve Explained, Central Limit Theorem — Why Averages Form Bells. Consider exploring them first for the best experience.
Take a Single Sample
We draw n observations from a population with true mean μ and known σ. The sample mean x̄ is our best estimate of μ — but it varies from sample to sample.
Build a Confidence Interval
Using x̄ ± z* · σ/√n we construct an interval around x̄. At 95% confidence, z* = 1.960. The interval either contains μ or it doesn't — there's no probability about it for any single interval.
Repeat: Draw Many Samples
Watch as we draw many samples and build a CI for each one. Green intervals contain the true μ; red ones miss it. Most are green — but not all.
Coverage Rate Converges
As we draw hundreds of intervals, the fraction containing μ converges toward 95%. This is what "95% confident" means: the procedure captures μ 95% of the time.
Change the Confidence Level
Switching from 95% to 99% makes every interval wider — more coverage, but less precision. At 80%, intervals shrink but more miss μ. The trade-off is coverage vs. precision.
Change the Sample Size — Then Estimate σ
Increasing n makes intervals narrower (SE = σ/√n shrinks), but the coverage rate stays the same. Now ask what if σ is unknown: choose "Sample s with z*" at n = 5 and only about 88% of "95%" intervals catch μ. Switch to "Sample s with t*" (t* = 2.776 for 4 degrees of freedom) and coverage returns to 95%.
How It Works — Step by Step
-
1
Take a Single Sample
We draw n observations from a population with true mean μ and known σ. The sample mean x̄ is our best estimate of μ — but it varies from sample to sample.
-
2
Build a Confidence Interval
Using x̄ ± z* · σ/√n we construct an interval around x̄. At 95% confidence, z* = 1.960. The interval either contains μ or it doesn't — there's no probability about it for any single interval.
-
3
Repeat: Draw Many Samples
Watch as we draw many samples and build a CI for each one. Green intervals contain the true μ; red ones miss it. Most are green — but not all.
-
4
Coverage Rate Converges
As we draw hundreds of intervals, the fraction containing μ converges toward 95%. This is what "95% confident" means: the procedure captures μ 95% of the time.
-
5
Change the Confidence Level
Switching from 95% to 99% makes every interval wider — more coverage, but less precision. At 80%, intervals shrink but more miss μ. The trade-off is coverage vs. precision.
-
6
Change the Sample Size — Then Estimate σ
Increasing n makes intervals narrower (SE = σ/√n shrinks), but the coverage rate stays the same. Now ask what if σ is unknown: choose "Sample s with z*" at n = 5 and only about 88% of "95%" intervals catch μ. Switch to "Sample s with t*" (t* = 2.776 for 4 degrees of freedom) and coverage returns to 95%.
Key Takeaways
It's About the Procedure, Not One Interval
"95% confident" means: if we repeated this sampling procedure many times, about 95% of the resulting intervals would contain the true mean μ.
Common Misconception
A 95% CI does NOT mean "there's a 95% probability the true mean is in this interval." The true mean is fixed — it's either in there or it isn't. The probability is about the method.
Width vs. Coverage Trade-off
Higher confidence levels produce wider intervals (more coverage, less precision). Lower levels give narrower intervals (more precise, but miss μ more often).
Estimating σ Costs Width
Increasing n makes intervals narrower — SE = σ/√n. When σ must be estimated by s, use t* with n − 1 degrees of freedom: it widens small-sample intervals just enough to keep the promised coverage, and it approaches z* as n grows.
Connected Concepts
Normal Distribution
Confidence intervals rely on the bell curve — the sampling distribution of the mean is normal.
Central Limit Theorem
The CLT guarantees that sample means are normally distributed, which makes CIs work.
Hypothesis Testing
If a CI does not contain the hypothesised value, you reject H₀ at that significance level.
Standard Deviation
The sample s estimates σ — and its sample-to-sample wobble is why small-n intervals need t* instead of z*.
Law of Large Numbers
The running coverage rate settles toward the confidence level for the same reason long-run averages settle.
Updated
- — Added a predict question on how the interval width changes with confidence, and a goal: draw 100 intervals at 80 %.
- — Added a σ-unknown mode (sample s with z* or t*), a "Predict the misses" challenge and two quiz questions.