Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Lesson 8 — Unsupervised Learning: Clustering & PCA

(customer segmentation, dimensionality reduction, and responsible interpretation)

Why this matters

In business and economics, we often do not begin with a clear target variable.

For example:

Unsupervised learning helps us:

Today’s mini case: customer segmentation

Today we use a synthetic customer dataset:

Each row is one customer.

The dataset includes variables such as:

What is unsupervised learning?

Today we focus on two tools:

  1. Clustering
    Group similar observations into clusters or segments.

  2. PCA
    Reduce many variables into a smaller number of components that summarize variation in the data.

Part A — Clustering for segmentation

What clustering does

Clustering groups observations based on similarity.

For customer segmentation, the idea is simple:

Customers who behave similarly should be placed in the same group.

For example, customers may differ in:

Typical business questions include:

Why scaling matters

K-means clustering is based on distance.

If one variable has a much larger numerical scale than another variable, it can dominate the distance calculation.

For example:

Without standardization, income may dominate the clustering simply because it has larger numbers.

Standardization converts each variable into a z-score:

This allows variables measured in different units to contribute more fairly to the clustering.

Choosing the number of clusters

K-means requires us to choose the number of clusters, usually called kk.

There is no single correct value of kk.

We can use:

  1. Elbow method
    Look at how inertia falls as kk increases.

  2. Silhouette score
    Measures how well-separated and compact the clusters are.

  3. Interpretability
    Do the clusters make sense?

  4. Usefulness
    Would the segmentation change business decisions?

Cluster labeling: the human step

After the algorithm assigns cluster numbers, we must interpret them.

Cluster numbers such as 0, 1, 2, and 3 have no meaning by themselves.

We need to inspect the cluster profiles:

Then we can assign meaningful labels such as:

Part B — PCA for visualization and dimensionality reduction

Why PCA exists

When we have many variables, patterns can be hard to see.

For example, in this dataset we have several customer characteristics:

It is difficult to visualize eight variables at once.

PCA creates new variables called principal components.

These components summarize variation in the original variables.

What we use PCA for today

Today we use PCA mainly for visualization.

After running k-means on the standardized variables, we use PCA to reduce the data to two dimensions:

Then we create a scatter plot:

This helps us see whether the clusters are clearly separated.

Interpreting PCA

Two things matter:

  1. Explained variance ratio
    How much variation is captured by each component?

  2. Loadings
    Which original variables contribute most to each component?

For example:

Mini-lab: customer segmentation in Python

In-class workflow

Step 1 — Load the data

We begin by loading the Excel file.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans
from sklearn.decomposition import PCA
from sklearn.metrics import silhouette_score

customers = pd.read_csv("customer_segmentation.csv")

customers.head()

Step 2 — Inspect the data

customers.shape
customers.info()
customers.describe()
customers["true_segment"].value_counts()

Remember: true_segment is included only for teaching. We will not use it for clustering.

Step 3 — Visualize a few variables

plt.figure()
plt.hist(customers["total_spend"], bins=30)
plt.title("Distribution of Total Spending")
plt.xlabel("Total spend")
plt.ylabel("Number of customers")
plt.show()

plt.figure()
plt.hist(customers["recency_days"], bins=30)
plt.title("Distribution of Recency")
plt.xlabel("Days since last purchase")
plt.ylabel("Number of customers")
plt.show()

plt.figure()
plt.scatter(customers["frequency"], customers["total_spend"], alpha=0.7)
plt.title("Spending and Purchase Frequency")
plt.xlabel("Purchase frequency")
plt.ylabel("Total spend")
plt.show()

Step 4 — Choose variables for clustering

We choose numeric variables that describe customer behavior and characteristics.

features = [
    "total_spend",
    "frequency",
    "recency_days",
    "avg_order_value",
    "discount_rate",
    "return_rate",
    "age",
    "income"
]

X = customers[features].copy()

X.head()

We do not include:

customer_id, because it is just an identifier. true_segment, because in real unsupervised learning we would not know it.

Step 5 — Standardize the variables

scaler = StandardScaler()

X_scaled = scaler.fit_transform(X)
X_scaled = pd.DataFrame(X_scaled, columns=features)
X_scaled.head()

Step 6 — Run k-means clustering

Start with four clusters.

kmeans = KMeans(n_clusters=4, random_state=42, n_init=20)
customers["cluster"] = kmeans.fit_predict(X_scaled)
customers[["customer_id", "cluster"] + features].head()

Check cluster sizes:

customers["cluster"].value_counts().sort_index()

Step 7 — Profile the clusters

cluster_profile = customers.groupby("cluster")[features].mean().round(2)
cluster_profile

A transposed version is often easier to read:

cluster_profile.T

Step 8 — Compare with the hidden true segments

This step is only possible because the dataset is synthetic.

pd.crosstab(customers["cluster"], customers["true_segment"])

With row percentages:

pd.crosstab(
    customers["cluster"],
    customers["true_segment"],
    normalize="index"
).round(2)

Step 9 — Choose the number of clusters: elbow method

inertias = []

K_range = range(2, 11)

for k in K_range:
    km = KMeans(n_clusters=k, random_state=42, n_init=20)
    km.fit(X_scaled)
    inertias.append(km.inertia_)

plt.figure()
plt.plot(list(K_range), inertias, marker="o")
plt.title("Elbow Method")
plt.xlabel("Number of clusters")
plt.ylabel("Inertia")
plt.show()

Step 10 — Choose the number of clusters: silhouette score

silhouette_scores = []

K_range = range(2, 11)

for k in K_range:
    km = KMeans(n_clusters=k, random_state=42, n_init=20)
    labels = km.fit_predict(X_scaled)
    score = silhouette_score(X_scaled, labels)
    silhouette_scores.append(score)

plt.figure()
plt.plot(list(K_range), silhouette_scores, marker="o")
plt.title("Silhouette Score by Number of Clusters")
plt.xlabel("Number of clusters")
plt.ylabel("Silhouette score")
plt.show()
pd.DataFrame({
    "k": list(K_range),
    "silhouette_score": silhouette_scores
}).round(3)

Step 11 — Visualize clusters using PCA

pca = PCA(n_components=2)

pca_components = pca.fit_transform(X_scaled)

customers["PC1"] = pca_components[:, 0]
customers["PC2"] = pca_components[:, 1]

customers[["PC1", "PC2", "cluster"]].head()

Check explained variance:

explained = pca.explained_variance_ratio_

print("Explained variance by PC1:", round(explained[0], 3))
print("Explained variance by PC2:", round(explained[1], 3))
print("Total explained variance:", round(explained.sum(), 3))

Plot clusters in PCA space:

plt.figure()
plt.scatter(
    customers["PC1"],
    customers["PC2"],
    c=customers["cluster"],
    alpha=0.7
)
plt.title("Customer Clusters Visualized with PCA")
plt.xlabel("Principal Component 1")
plt.ylabel("Principal Component 2")
plt.show()
Step 12 — Inspect PCA loadings
loadings = pd.DataFrame(
    pca.components_.T,
    columns=["PC1", "PC2"],
    index=features
)

loadings.round(3)

Step 13 — Give business labels to clusters

After inspecting the cluster profile, assign labels manually.

Start with a cautious version:

cluster_labels = {
    0: "Cluster 0: inspect profile",
    1: "Cluster 1: inspect profile",
    2: "Cluster 2: inspect profile",
    3: "Cluster 3: inspect profile"
}

customers["cluster_label"] = customers["cluster"].map(cluster_labels)
customers[["customer_id", "cluster", "cluster_label"]].head()

After looking at the profiles, revise the labels. For example:

cluster_labels = {
    0: "Occasional Customers",
    1: "Loyal High-Value Customers",
    2: "Discount-Driven Customers",
    3: "At-Risk Customers"
}

customers["cluster_label"] = customers["cluster"].map(cluster_labels)
customers[["customer_id", "cluster", "cluster_label"]].head()

Step 14 — Summarize labeled clusters

summary = customers.groupby("cluster_label").agg(
    n_customers=("customer_id", "count"),
    avg_total_spend=("total_spend", "mean"),
    avg_frequency=("frequency", "mean"),
    avg_recency_days=("recency_days", "mean"),
    avg_discount_rate=("discount_rate", "mean"),
    avg_return_rate=("return_rate", "mean"),
    avg_income=("income", "mean")
).round(2)

summary

Step 15 — Create a standardized cluster profile

This makes it easier to see which clusters are above or below average on each feature.

standardized_profile = X_scaled.copy()
standardized_profile["cluster_label"] = customers["cluster_label"]

standardized_profile.groupby("cluster_label")[features].mean().round(2)

Interpretation:

Visual communication

For clustering, useful outputs include:

For PCA, useful outputs include:

Responsible interpretation and ethics

Customer segmentation can be useful, but it can also be misused.

Possible risks include:

Bridge to Lesson 9: decision trees

Lesson 8 uses the customer data for unsupervised learning.

The question is:

The question becomes:

AI check: responsible use for clustering and PCA

Good prompt examples:

Bad prompt example:

This is too vague and invites overinterpretation.

Review questions

Notes
Lesson 7 — Data Visualization for Communication
Notes
Lesson 9 — Decision Trees + Text-as-Data (NLP) + Ethics