Correlation & Regression Basics
How strongly do two variables move together — and can we predict one from the other? Plus the most important statistical caution: correlation is not causation.
Learning Objectives
- Define correlation and interpret the Pearson coefficient (−1 to +1).
- Explain correlation vs causation — and why it matters.
- Describe linear regression and interpret slope and intercept.
- Explain R-squared and simple vs multiple regression.
- Recognize confounding variables.
1. What Is Correlation?
Correlation measures the strength and direction of a linear relationship between two numeric variables. The most common measure is the Pearson correlation coefficient, which ranges from −1 to +1.
| Value | Meaning |
|---|---|
+1 | perfect positive — both move up together |
0 | no linear relationship |
−1 | perfect negative — one up, other down |
import pandas as pd
df["price"].corr(df["units_sold"]) # e.g. -0.85 → strong negative2. Correlation vs Causation — the #1 Trap
Correlation means two variables move together. Causation means one actually causes the other. Just because two things are correlated does not mean one caused the other.
To establish causation you need experiments (like A/B tests) or careful study design — not just a high correlation.
3. Linear Regression
Linear regression fits a straight line to predict a dependent variable (y) from an independent variable (x):
# y = slope * x + intercept
import numpy as np
slope, intercept = np.polyfit(x, y, 1)| Term | Meaning |
|---|---|
| Slope (m) | how much y changes for a 1-unit increase in x |
| Intercept (b) | the value of y when x = 0 |
sales = 2.5 × ad_spend + 1000, then each extra unit of ad spend is
associated with 2.5 more sales, and sales would be 1000 with zero ad spend.
4. R-Squared (R²)
R² measures how much of the variation in y is explained by the model. It ranges from 0 to 1:
R² = 0.9— the model explains 90% of the variation (a good fit).R² = 0.1— the model explains only 10% (a poor fit).
A high R² means the line fits well, but it doesn't by itself prove causation.
5. Simple vs Multiple Regression
| Simple | Multiple | |
|---|---|---|
| Predictors | one independent variable | two or more |
| Example | sales ~ ad_spend | sales ~ ad_spend + price + region |
Multiple regression lets you control for other factors — reducing the risk of a spurious single-variable relationship.
6. Interview Questions (with Model Answers)
The correlation and regression questions interviewers ask. Self-test before revealing.
IQ1. What is correlation, and how is it measured?
Model answer: "Correlation measures the strength and direction of a linear relationship between two numeric variables. The Pearson coefficient ranges from -1 to +1 — positive means they move together, negative means opposite, zero means no linear relationship."
IQ2. What does a correlation of -0.9 mean?
Model answer: "A strong negative relationship — as one variable goes up, the other tends to go down. The closer to -1 (or +1), the stronger the linear relationship."
IQ3. What's the difference between correlation and causation?
Model answer: "Correlation means two variables move together; causation means one actually drives the other. A high correlation doesn't prove causation — a third confounding variable may drive both."
IQ4. Give an example of correlation without causation.
Model answer: "Ice-cream sales and drowning incidents are correlated, but ice cream doesn't cause drowning — hot weather drives both. That's a confounding variable."
IQ5. What is linear regression?
Model answer: "Linear regression fits a straight line to predict a dependent variable from one or more independent variables — y = mx + b. It's the simplest predictive model."
IQ6. How do you interpret the slope and intercept?
Model answer: "The slope is how much y changes for a one-unit increase in x; the intercept is the predicted y when x is zero. In sales = 2.5×ad_spend + 1000, each ad-spend unit adds 2.5 sales."
IQ7. What is R-squared?
Model answer: "R² measures the proportion of variation in y explained by the model, from 0 to 1. An R² of 0.9 means the model explains 90% of the variation — a good fit."
IQ8. What's the difference between simple and multiple linear regression?
Model answer: "Simple regression uses one independent variable; multiple regression uses two or more. Multiple regression lets me control for other factors, reducing spurious single-variable relationships."
IQ9. What is a confounding variable?
Model answer: "A confounding variable is a hidden factor that influences both the variables I'm studying, creating a false appearance of a direct relationship between them."
Hands-On Project: Correlation & Regression in Python
Compute correlation and a simple regression on two variables.
Steps
- Create
ad_spend = [10, 20, 30, 40, 50]andsales = [25, 55, 80, 105, 130]. - Compute the Pearson correlation between them.
- Fit a linear regression line and print the slope and intercept.
- Predict sales for an ad spend of 35.
View Solution / Walkthrough
import numpy as np
import pandas as pd
ad_spend = [10, 20, 30, 40, 50]
sales = [25, 55, 80, 105, 130]
# 2. Correlation
corr = pd.Series(ad_spend).corr(pd.Series(sales))
print(corr) # ≈ 0.999 (very strong positive)
# 3. Slope and intercept
slope, intercept = np.polyfit(ad_spend, sales, 1)
print(slope, intercept) # ≈ 2.6, 0.0
# 4. Predict sales for ad_spend = 35
pred = slope * 35 + intercept
print(pred) # ≈ 91Key Takeaways
Correlation ranges −1 to +1: strength and direction of a linear relationship.
Correlation ≠ causation — a confounder may drive both.
Slope = change in y per unit x; intercept = y when x = 0.
R² = proportion of variance explained (0 to 1).
Multiple regression controls for confounders; simple doesn't.
Objective Questions — Test Your Understanding
Q1. The Pearson correlation coefficient ranges from…
Q2. A correlation of −0.9 indicates…
Q3. "Correlation does not imply ___"
Q4. In the equation y = mx + b, what does m represent?
Q5. What does R-squared (R²) measure?