Linear Regression — The Line of Best Fit

Play

Finding the line that best fits your data

Mathematics Statistics intermediate 7 min

This concept builds on Slope — Rise Over Run: Finding the Slope of a Line. Consider exploring it first for the best experience.

Step 1 of 4

A Cloud of Points

Real-world data is messy — measurements scatter around a trend instead of lying on a perfect line. Watch the data points appear and notice the upward pattern.

The Best-Fit Line

Ordinary Least Squares (OLS) finds the unique line ŷ = β̂₀ + β̂₁x that minimizes the total squared vertical distance from every point to the line.

Residuals — The Gaps

Each vertical gap between a point and the line is a residual. OLS minimizes the sum of these squared gaps. Toggle residuals on to see them.

R² — Goodness of Fit

R² tells you what fraction of the variation in y is "explained" by x. R² = 1 means a perfect fit; R² ≈ 0 means x tells you almost nothing about y.

Play Best Fit A game about Linear Regression — The Line of Best Fit

How It Works — Step by Step

Step 1 of 4
  1. 1

    A Cloud of Points

    Real-world data is messy — measurements scatter around a trend instead of lying on a perfect line. Watch the data points appear and notice the upward pattern.

  2. 2

    The Best-Fit Line

    Ordinary Least Squares (OLS) finds the unique line ŷ = β̂₀ + β̂₁x that minimizes the total squared vertical distance from every point to the line.

  3. 3

    Residuals — The Gaps

    Each vertical gap between a point and the line is a residual. OLS minimizes the sum of these squared gaps. Toggle residuals on to see them.

  4. 4

    R² — Goodness of Fit

    R² tells you what fraction of the variation in y is "explained" by x. R² = 1 means a perfect fit; R² ≈ 0 means x tells you almost nothing about y.

Key Takeaways

Minimizing Squared Residuals

The regression line minimizes the sum of squared vertical distances (residuals). This is why it's called "least squares".

R² Measures Explanatory Power

R² tells you what fraction of the variation in y is explained by x. It equals the square of the correlation coefficient r.

Slope and Correlation Are Connected

The regression slope β̂₁ = r · (sᵧ/sₓ). Correlation gives direction and strength; the standard deviation ratio scales it to real units.

Outliers Have Leverage

One outlier can dramatically shift the regression line. Always visualize your data — summary statistics alone can be misleading.

The Line Passes Through the Means

Because β̂₀ = ȳ − β̂₁x̄, the least-squares line always goes through the mean point (x̄, ȳ). Try the "Guess the Line" challenge: pivoting your line around (x̄, ȳ) is the fastest way to shrink your SSE.

Predict Inside the Data

Plugging an x into ŷ = β̂₀ + β̂₁x gives a prediction. Inside the observed x-values that is interpolation; outside them it is extrapolation, which assumes the trend continues where no data was measured. Turn on "Predict at x" and drag a dot to see far-out predictions swing much more than ones near x̄.

Added
Updated
  • — New 'Predict at x' mode: slide x to read ŷ off the line and see when a prediction becomes extrapolation.