Lesson 6 — Correlation, Simple & Multiple Regression

(interpreting relationships in business and economics)

Why this matters¶

Regression is one of the most widely used tools in business, economics, and social science.

It helps us quantify relationships such as:

Are richer countries associated with different family or demographic outcomes?
Are prices associated with lower demand?
Are marketing expenditures associated with higher sales?
Are customer characteristics associated with churn or satisfaction?

But regression is also easy to misuse, especially when we confuse association with causation.

Where regression fits in the course¶

Regression is therefore a bridge between traditional statistics, econometrics, and modern machine learning.

The language may differ, but the basic idea is similar:

Statistics / Econometrics	Machine Learning
Dependent variable	Target / outcome
Independent variable	Feature / predictor
Regression model	Supervised learning model
Interpretation	Prediction and/or explanation

The regression mindset: from question to model¶

Before running a regression, be clear about four things:

Outcome $Y$
What are we trying to explain or predict?
Predictor $X$
What variable might be related to the outcome?
Unit of observation
Are we analyzing customers, firms, countries, country-years, or something else?
Interpretation goal
Are we describing a relationship, predicting an outcome, or making a causal claim?

Today’s running example: divorce and income¶

We use:

divorce_raw.csv

The dataset contains country-level information on:

country name and country code,
year,
marriage rate,
divorce rate,
GDP growth,
GDP per capita,
population.

For today’s class, we focus on a single cross-section:

Unit of observation: country
Year: 1992
Outcome $Y$ : divorce-marriage rate
d_rate = divorce_rate / marriage_rate * 1000
Main predictor $X$ : GDP per capita, rescaled
income_pc = gdp_pc / 1000
Control variable: GDP growth
gdp_gr

Step 0 — Start with a picture¶

Before any equation, draw the relationship.

A scatter plot helps us see:

whether there is an upward or downward pattern,
whether the relationship looks weak or strong,
whether there are outliers,
whether a straight line is reasonable.

Step 1 — Correlation¶

Correlation is a standardized measure of linear association.

It ranges from -1 to +1.

A positive correlation means high values of one variable tend to be associated with high values of the other.
A negative correlation means high values of one variable tend to be associated with low values of the other.
A correlation near zero means there is little linear association.

Correlation is useful as a warm-up, but it does not tell us how much $Y$ changes when $X$ changes in real units.

Step 2 — Simple regression¶

A simple regression estimates a line:

$Y = \beta_0 + \beta_1 X + \varepsilon$

where:

$Y$ is the outcome,
$X$ is the predictor,
$\beta_0$ is the intercept,
$\beta_1$ is the slope,
$\varepsilon$ captures everything else not included in the model.

In today’s example:

$Y$ is d_rate
$X$ is income_pc

The model is:

$d_rate = \beta_0 + \beta_1 income\_pc + \varepsilon$

Interpretation template¶

A one-unit increase in income_pc, meaning a $1,000 increase in GDP per capita, is associated with a$ \beta_1$ change in d_rate, on average, in this sample.

Because d_rate is measured as divorces per 1,000 marriages, the coefficient is interpreted in that unit.

Step 3 — Multiple regression¶

Multiple regression adds more than one predictor:

$Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \varepsilon$

In today’s example:

$d_rate = \beta_0 + \beta_1 income\_pc + \beta_2 gdp\_gr + \varepsilon$

Now $\beta_1$ is interpreted as the association between income_pc and d_rate, holding gdp_gr constant.

Why add a control?¶

We add controls when we think another variable may be related to both the outcome and the main predictor.

For example, economic conditions may be related to both income and family formation patterns. Adding gdp_gr is a simple way to begin asking whether the income association changes when we account for macroeconomic growth.

Confounding: a simple intuition¶

Suppose we find that villages with more temples also have more crime.

Does that mean temples cause crime?

Probably not.

A more likely explanation is that larger villages have both:

more temples,
and more crime.

Village size is a confounder because it is related to both variables.

How to read regression output¶

When reading regression output, focus on four things.

1. Coefficients¶

Ask:

Is the coefficient positive or negative?
How large is it?
What are the units?
Does the sign make sense?

2. Standard errors and confidence intervals¶

Ask:

How precise is the estimate?
Is the estimate uncertain?

3. p-values¶

Ask:

Is the association statistically distinguishable from zero under the model assumptions?
But do not treat the p-value as the only thing that matters.

4. R-squared¶

Ask:

How much variation in $Y$ is explained by the model in this sample?
Is the model useful for the question at hand?

Regression and causality¶

Regression describes a relationship.

Causal interpretation requires stronger assumptions or a research design.

Common threats include:

omitted variables,
reverse causality,
measurement error,
selection bias,
bad controls,
extrapolation.

Mini-lab: regression in Google Colab¶

In-class workflow¶

Step 1 — Load packages and data¶

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import statsmodels.api as sm

df = pd.read_csv("divorce_raw.csv")

df.head()
Step 2 — Filter to one year
df92 = df[df["year"] == 1992].copy()

df92.head()

We use .copy() because we do not want to accidentally change the original dataset.

Step 3 — Check the variables
df92.info()
df92.describe()

Make sure the key variables are numeric:

cols = ["marriage_rate", "divorce_rate", "gdp_gr", "gdp_pc", "population"]

for col in cols:
    df92[col] = pd.to_numeric(df92[col], errors="coerce")
Step 4 — Create analysis variables
df92["income_pc"] = df92["gdp_pc"] / 1000
df92["d_rate"] = df92["divorce_rate"] / df92["marriage_rate"] * 1000

Remove impossible or missing values:

df92 = df92.replace([np.inf, -np.inf], np.nan)
df92 = df92.dropna(subset=["income_pc", "d_rate", "gdp_gr"]).copy()
df92 = df92[df92["marriage_rate"] > 0].copy()

Check the new variables:

df92[["country", "income_pc", "d_rate", "gdp_gr"]].head()

Step 5 — Start with a scatter plot

plt.figure()
plt.scatter(df92["income_pc"], df92["d_rate"], alpha=0.7)
plt.title("Divorce-Marriage Rate and Income, 1992")
plt.xlabel("GDP per capita, thousands of dollars")
plt.ylabel("Divorces per 1,000 marriages")
plt.show()

Step 6 — Correlation

corr = df92[["income_pc", "d_rate"]].corr().iloc[0, 1]
corr

Interpretation:

The correlation tells us the direction and strength of the linear relationship between income and the divorce-marriage rate. It does not imply causation.

Step 7 — Simple regression

X = sm.add_constant(df92["income_pc"])
y = df92["d_rate"]

model_simple = sm.OLS(y, X).fit()
model_simple.summary2()

Interpret the coefficient on income_pc:

A one-unit increase in income_pc, meaning a $1,000 increase in GDP per capita, is associated with a change of coefficient divorces per 1,000 marriages, on average, in this sample.

Step 8 — Multiple regression

X2 = sm.add_constant(df92[["income_pc", "gdp_gr"]])
y2 = df92["d_rate"]

model_multi = sm.OLS(y2, X2).fit()
model_multi.summary2()

Compare the coefficient on income_pc in the simple and multiple regressions.

Did the sign change?
Did the magnitude change?
Did the uncertainty change?
What does “holding GDP growth constant” mean here?

Step 9 — Residual diagnostic plot (optional)

fitted = model_multi.fittedvalues
residuals = model_multi.resid

plt.figure()
plt.scatter(fitted, residuals, alpha=0.7)
plt.axhline(0, linestyle="--")
plt.title("Residuals vs Fitted Values")
plt.xlabel("Fitted values")
plt.ylabel("Residuals")
plt.show()

This is a simple diagnostic plot.

It helps us see whether residuals are centered around zero and whether there are obvious patterns.

Step 10 — Write a short manager memo

Write 5–7 lines.

Use this structure:

Headline: What is the main pattern?
Evidence: What do the scatter plot and regression suggest?
Control: Did the result change after adding GDP growth?
Caveat: Why should we avoid causal language?

Example:

Countries with higher GDP per capita appear to have different divorce-marriage patterns in 1992. The simple regression suggests that income is associated with the divorce-marriage rate, but this is only a descriptive relationship. After adding GDP growth, the coefficient should be interpreted as the association holding growth constant. The result should not be interpreted causally because many social, legal, demographic, and cultural factors are not included in the model.

AI check: responsible use

Good prompt:

I ran a regression of d_rate on income_pc and gdp_gr. d_rate is divorces per 1,000 marriages. income_pc is GDP per capita divided by 1,000. Please help me write a cautious interpretation of the coefficients.

Avoid causal language.

Bad prompt:

Prove that income causes divorce. In-class checkpoints Load divorce_raw.csv. Filter to year == 1992. Create: income_pc = gdp_pc / 1000 d_rate = divorce_rate / marriage_rate * 1000 Make a scatter plot of d_rate against income_pc. Compute the correlation between d_rate and income_pc. Fit the simple regression: d_rate ~ income_pc. Fit the multiple regression: d_rate ~ income_pc + gdp_gr. Compare the coefficient on income_pc across the two models. Make one residual diagnostic plot. Write a short manager memo with one caveat. Submission after class

Submit either:

Upload pdf to link []

Your submission should include:

A scatter plot. Simple and multiple regression output. A short interpretation of the coefficient on income_pc. A short manager memo. One clear caveat about causality or omitted variables.

Review questions

What is the difference between correlation and regression?
Why is regression a form of supervised learning?
What does the coefficient on income_pc mean in the simple regression?
What changes when gdp_gr is added as a control variable?
Why should we avoid saying that income “causes” divorce patterns based only on this regression?
What is extrapolation, and why is it risky?