Linear Regression — The Line of Best Fit
PlayFinding the line that best fits your data
This concept builds on Slope — Rise Over Run: Finding the Slope of a Line. Consider exploring it first for the best experience.
A Cloud of Points
Real-world data is messy — measurements scatter around a trend instead of lying on a perfect line. Watch the data points appear and notice the upward pattern.
The Best-Fit Line
Ordinary Least Squares (OLS) finds the unique line ŷ = β̂₀ + β̂₁x that minimizes the total squared vertical distance from every point to the line.
Residuals — The Gaps
Each vertical gap between a point and the line is a residual. OLS minimizes the sum of these squared gaps. Toggle residuals on to see them.
R² — Goodness of Fit
R² tells you what fraction of the variation in y is "explained" by x. R² = 1 means a perfect fit; R² ≈ 0 means x tells you almost nothing about y.
How It Works — Step by Step
-
1
A Cloud of Points
Real-world data is messy — measurements scatter around a trend instead of lying on a perfect line. Watch the data points appear and notice the upward pattern.
-
2
The Best-Fit Line
Ordinary Least Squares (OLS) finds the unique line ŷ = β̂₀ + β̂₁x that minimizes the total squared vertical distance from every point to the line.
-
3
Residuals — The Gaps
Each vertical gap between a point and the line is a residual. OLS minimizes the sum of these squared gaps. Toggle residuals on to see them.
-
4
R² — Goodness of Fit
R² tells you what fraction of the variation in y is "explained" by x. R² = 1 means a perfect fit; R² ≈ 0 means x tells you almost nothing about y.
Key Takeaways
Minimizing Squared Residuals
The regression line minimizes the sum of squared vertical distances (residuals). This is why it's called "least squares".
R² Measures Explanatory Power
R² tells you what fraction of the variation in y is explained by x. It equals the square of the correlation coefficient r.
Slope and Correlation Are Connected
The regression slope β̂₁ = r · (sᵧ/sₓ). Correlation gives direction and strength; the standard deviation ratio scales it to real units.
Outliers Have Leverage
One outlier can dramatically shift the regression line. Always visualize your data — summary statistics alone can be misleading.
The Line Passes Through the Means
Because β̂₀ = ȳ − β̂₁x̄, the least-squares line always goes through the mean point (x̄, ȳ). Try the "Guess the Line" challenge: pivoting your line around (x̄, ȳ) is the fastest way to shrink your SSE.
Predict Inside the Data
Plugging an x into ŷ = β̂₀ + β̂₁x gives a prediction. Inside the observed x-values that is interpolation; outside them it is extrapolation, which assumes the trend continues where no data was measured. Turn on "Predict at x" and drag a dot to see far-out predictions swing much more than ones near x̄.
Connected Concepts
Correlation
Correlation measures the strength and direction of a linear relationship — regression fits the line.
Slope
The regression slope β̂₁ is rise-over-run applied to a cloud of points instead of two.
Normal Distribution
Residuals in a good regression model are normally distributed around zero.
Gradient Descent
SSE is a loss function over (slope, intercept) — gradient descent walks downhill on it to the same least-squares line.
Updated
- — New 'Predict at x' mode: slide x to read ŷ off the line and see when a prediction becomes extrapolation.