Multiple Linear Regression: Modelling Several Predictors at Once

House price depends on more than just square footage — location, age, number of bedrooms, and condition all matter too. Simple linear regression can only handle one predictor at a time. Multiple linear regression extends the same underlying idea to several predictors simultaneously, letting a single model account for many factors at once and separate each one's individual contribution.

The Model

Multiple regression is directly built on the same least-squares logic covered in simple linear regression and Pearson correlation — the extension to several predictors at once is what makes it useful for the genuinely multi-factor questions most real-world data actually presents.

Each predictor (x₁, x₂, ... xₖ) gets its own coefficient (b₁, b₂, ... bₖ), estimated simultaneously using the same least-squares principle as simple linear regression — minimizing the total squared difference between predicted and actual y values, just now across a multi-dimensional space of predictors rather than a single line.

Interpreting Coefficients: "Holding All Else Constant"

This is the single most important — and most often misunderstood — difference from simple regression. In multiple regression, each coefficient represents the predicted change in y for a one-unit increase in that specific predictor, while holding every other predictor in the model constant. This "holding constant" interpretation is what lets multiple regression isolate one variable's individual effect from the others, even when the predictors are themselves correlated with each other in the real data.

Example: a house price model with predictors square footage and number of bedrooms might show a coefficient on bedrooms that's small or even negative — not because more bedrooms genuinely reduce value, but because once square footage is already accounted for, adding a bedroom by subdividing existing space (rather than adding square footage) can actually make individual rooms smaller and less desirable. The coefficient reflects bedrooms' effect independent of size, which can be a genuinely different, less intuitive story than bedrooms' effect considered alone.

Worked Example (Conceptual)

Problem: A fitted model predicts salary from years of experience and education level (years of schooling): Salary = 28,000 + 2,100×(experience) + 3,400×(education years).

Interpretation: holding education constant, each additional year of experience is associated with a predicted $2,100 increase in salary. Holding experience constant, each additional year of education is associated with a predicted $3,400 increase. The intercept, $28,000, is the model's prediction for someone with zero years of both experience and education — often not a practically meaningful value on its own, exactly as with simple regression's intercept, but mathematically necessary to fit the model.

Multicollinearity: When Predictors Are Too Correlated With Each Other

Multicollinearity occurs when two or more predictors are themselves strongly correlated — like a person's height in centimetres and height in inches included as separate predictors in the same model (an extreme, obvious example). When predictors overlap heavily, the model struggles to separate their individual effects, producing unstable, sometimes counterintuitive coefficients (like a variable that's clearly positively related to the outcome showing a negative coefficient) and inflated standard errors on those coefficients.

Adjusted R²: A Fairer Comparison Across Models

Plain R² always increases (or stays the same) as more predictors are added to a model, even predictors that are genuinely useless — purely because more parameters give the model more flexibility to fit the specific data at hand, a phenomenon called overfitting. Adjusted R² corrects for this by penalizing the addition of predictors that don't meaningfully improve the model's fit, making it the more honest metric for comparing models with different numbers of predictors, or deciding whether adding a new variable is genuinely worth the added complexity.

Categorical Predictors in Multiple Regression

Categorical variables (like region, or product category) can be included using "dummy" or "indicator" coding — converting a category with k levels into k−1 binary (0/1) variables, each representing membership in one category relative to a chosen baseline. This lets multiple regression incorporate both numeric and categorical predictors within the same unified model, a technique that extends the framework well beyond purely numeric relationships.

Real-World Uses

Real estate valuation: automated home-value estimates routinely use multiple regression (or more sophisticated extensions of it) with dozens of predictors — square footage, location, age, renovation status, school district, and more — simultaneously.

Economics: estimating how multiple factors (interest rates, unemployment, consumer confidence) jointly influence an economic outcome relies on multiple regression to separate each factor's individual contribution.

Medical research: controlling for confounding variables (age, sex, pre-existing conditions) while estimating a treatment's effect is a standard application — multiple regression lets researchers isolate the treatment's effect independent of these other factors.

Common Mistakes

Interpreting a coefficient as if other variables didn't exist. This is the single most common misreading of multiple regression output — a coefficient's meaning is inherently tied to "holding everything else in the model constant," and dropping that qualifier when explaining results to others can seriously misrepresent what the model actually shows.

Adding predictors without checking multicollinearity. Throwing many correlated predictors into a model without checking VIF or similar diagnostics can produce a model with individually unstable, hard-to-interpret coefficients, even when the model's overall predictive accuracy looks fine.

Confusing a strong R² with a causally correct model. Exactly as with simple regression, a multiple regression model with a high R² demonstrates a strong statistical association across the included predictors — it doesn't establish that any of them causally drive the outcome, especially when the predictors themselves weren't experimentally manipulated.

Comparing R² (not adjusted R²) across models with different numbers of predictors. Since plain R² mechanically increases with more predictors regardless of their genuine usefulness, comparing raw R² between a 3-predictor and a 10-predictor model is misleading — adjusted R² is the fairer basis for that comparison.

Model Selection: Choosing Which Predictors to Include

With many candidate predictors available, deciding which to actually include in a final model is itself a genuine methodological challenge. Common approaches include forward selection (starting with no predictors and adding the most useful one at each step), backward elimination (starting with all candidates and removing the least useful), and more principled approaches based on information criteria (like AIC or BIC) that explicitly balance model fit against complexity, similar in spirit to adjusted R²'s penalty for unnecessary predictors. Purely automated selection procedures carry real risks — including overfitting to the specific sample and producing a model that doesn't generalize well — which is why domain knowledge about which predictors are theoretically plausible remains an essential complement to any automated selection method, not a step to skip in favour of letting an algorithm decide alone.