Part IV — Clustering

Chapter 19 · K-Means Clustering

Part IV removes the scaffolding: the labels. Nobody tells you who the buyers are — the data must reveal its own groups. K-Means is the classic answer: pick k centres, assign every point to its nearest, move the centres, repeat. This chapter runs the loop by hand, chooses k with the elbow and the silhouette (both peak at 5, here), measures the algorithm's init-fragility with a real experiment — and ends with five named customer segments a marketing team could act on Monday morning.

📖 Chapter 19 of 33 ⏱ ~85 min read Mostly Core Stable 📝 Quiz · 30 MCQs

🎯 Learning Objectives

Run the K-Means assign–update loop by hand on a six-point example, computing WCSS at each step
Choose k with the elbow method — and defend the choice with the silhouette coefficient (added beyond the source)
Explain initialisation sensitivity from a real seed experiment: k-means++, random, and the role of n_init
State K-Means' assumptions (spherical clusters, comparable units, k in advance) and its scaling obligations
Read and name the five real segments of the mall dataset, attaching a business action to each
Explain why Part IV has no confusion matrix — and what replaces it

1. No Labels, No Judge

Everything since Chapter 4 had a supervisor: a target column — salary, profit, purchased — against which every prediction was graded. Chapter 18's metrics were the last act of that contract. Part IV tears it up. Clustering asks the data alone to reveal its structure: which customers travel together? The evaluation consequence is immediate and strange — no labels means no confusion matrix, no accuracy, no recall. Judgement shifts from "did it match truth?" to "is this structure real and useful?" — measured by internal shape (this chapter's silhouette) and business meaning (this chapter's ending).

The running dataset: Mall_Customers.csv — 200 mall customers with Age, Genre, Annual Income (₹15k–137k), and a Spending Score (1–100) the mall computed from purchase behaviour. The business question: which customer groups exist, and what should marketing do with each? The source uses Income × Spending — the two behavioural axes — exactly as a segmentation consultant would.

📝

Note — unsupervised ≠ easier: with no target to be right about, the hard decisions move earlier (which features define "similarity"?) and later (is k = 5 real, or an artefact?). K-Means is simple; trusting its clusters is where the professionalism lives.

2. The Algorithm, by Hand

K-Means minimises the within-cluster sum of squares (WCSS) — the total squared distance from every point to its cluster's centroid:

WCSS = Σ_k Σ_{x ∈ C_k} ‖x − μ_k‖²

μ_k = centroid (mean) of cluster k. sklearn calls it inertia_. Lloyd's algorithm finds a (local) minimum by alternating two steps until nothing moves: Assign — each point joins its nearest centroid (Euclidean). Update — each centroid moves to the mean of its members.

2.1 Six points, one full run

Spending scores of six customers: 2, 3, 4, 10, 11, 12. Set k = 2; seed the centroids at 2 and 12 (endpoints, for hand-friendliness).

Step	Action	Result
Assign 1	each point → nearer centroid	{2,3,4} → 2 · {10,11,12} → 12
Update 1	centroids ← member means	μ₁ = (2+3+4)/3 = 3 · μ₂ = (10+11+12)/3 = 11
Assign 2	re-check every point	no point switches — {2,3,4} vs {10,11,12} stands
—	nothing moved → converged	final WCSS below

WCSS = [(2−3)² + (3−3)² + (4−3)²] + [(10−11)² + (11−11)² + (12−11)²] = (1+0+1) + (1+0+1) = 4

Verified against sklearn with these exact seeds: labels [0,0,0,1,1,1], centroids 3 and 11, inertia 4.0 — to the decimal. That is the whole algorithm; 200 customers and 2 dimensions are the same loop, iterated by machine. Two properties worth carrying forward: WCSS decreases (or holds) every step — convergence is guaranteed — but which minimum you land in depends on the seeding, a fragility Section 5 measures with real numbers.

3. Choosing k: the Elbow

The awkward parameter arrives on day one: how many clusters? The elbow method's logic: adding a cluster always shrinks WCSS (k = n gives WCSS = 0 — every point its own cluster), so look for the k where the improvement stops being dramatic — the arm's bend. The real WCSS values, k-means++ seeding:

k	1	2	3	4	5	6	7	8	9	10
WCSS	269,981	181,364	106,348	73,680	44,449	37,234	30,241	25,036	21,917	20,072
Drop	—	88,618	75,015	32,668	29,232	7,215	6,993	5,204	3,120	1,845

Figure 19.1 — The real elbow: drops of ~29k per extra cluster suddenly collapse to ~7k after k = 5 — the bend in the arm.

The drop column makes the argument precise: through k = 5, each new cluster buys 29k–89k of WCSS; from k = 6 on, merely 2k–7k. The arm bends at five. But read the method's fine print honestly: "dramatic" is a judgement, and some datasets give a forearm instead of an elbow. The next section supplies the second opinion — absent from the source code, and standard professional practice.

4. Silhouette: the Second Opinion

The silhouette coefficient scores each point's situation from −1 to +1 — no k needed in advance, no WCSS's always-improves trap:

s(i) = (b(i) − a(i)) / max(a(i), b(i))

a(i) — cohesion: mean distance from point i to its own cluster's other members
b(i) — separation: mean distance to the nearest other cluster's members
s ≈ +1: snug in its home, far from neighbours · s ≈ 0: on a border · s < 0: probably in the wrong cluster

The real silhouette scores, averaged over all 200 customers, per k:

k	2	3	4	5	6	7	8	9	10
Silhouette	0.297	0.468	0.493	0.554	0.540	0.529	0.455	0.456	0.441

Two independent instruments, one verdict: the silhouette peaks at exactly the elbow's k = 5 (0.554), then declines — k = 6's extra boundary slices a healthy cluster (0.540) without adding real separation. When elbow and silhouette agree, the choice of k carries evidence rather than taste; when they disagree, trust neither blindly — plot the clusters and bring business sense (Section 7's discipline).

💡

Pro Tip: the average silhouette can hide a rogue cluster — one cluster at −0.1 inside an average of 0.4 is a confession. Inspect per-cluster silhouette scores (silhouette_samples) before shipping segments, exactly as Chapter 10's residual audit looked past a single average.

5. Initialisation & Scaling

5.1 The seeding lottery, measured

Lloyd's algorithm finds a local WCSS minimum — and which one depends on where the centroids start. We ran the real experiment: k = 5, a single initialisation per run, ten different seeds:

Init (n_init = 1)	Best WCSS	Worst WCSS	Spread
`random`	44,448	89,897	45,449
`k-means++`	44,448	66,674	22,226

Three lessons in one table. Unlucky seeds can double the WCSS — same algorithm, same data, different answer. k-means++ (seed the first centroid randomly, then bias later seeds toward far-away points) halves the damage. And the real cure is restarts: n_init=10 — the classic default — runs the lottery ten times and keeps the best, which is how every number in this chapter was computed. Note that modern sklearn's n_init='auto' may run a single k-means++ pass; our experiment shows why pinning n_init=10 remains the careful habit.

5.2 Scaling — luck, not law

K-Means assigns by distance, so Chapter 3's rule applies: incomparable units silence features. The source's raw features escape by luck — Income (15–137) and Spending (1–99) happen to live on similar ranges, and standardising barely moves the answer (silhouette 0.5539 raw vs 0.5547 scaled; identical segment sizes). But run the same code with income in raw rupees (15,000–137,000) and Spending would vanish from every distance. The source skipped the scaler and got away with it; the professional default remains Chapter 3's: scale, unless you can say why not.

⚠

Warning — the shape assumption: K-Means draws spherical clusters of similar spread, because centroids and squared Euclidean distance encode exactly that geometry. Concentric rings, moons and elongated bands get carved into pie slices — wrong, confidently. Plot the data first; if the shape isn't blobby, reach for different structure-finders (Chapter 20's hierarchy, or spectral/DBSCAN beyond this course).

📋 Stable content — Reviewed: August 2026

6. Implementation in Python

import numpy as np
import pandas as pd

dataset = pd.read_csv('Mall_Customers.csv')
X = dataset.iloc[:, [3, 4]].values      # Annual Income, Spending Score

# The elbow sweep (source) + the silhouette second opinion
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score

wcss, sil = [], []
for k in range(1, 11):
    km = KMeans(n_clusters=k, init='k-means++', random_state=42,
                n_init=10).fit(X)
    wcss.append(km.inertia_)
    if k >= 2:
        sil.append(silhouette_score(X, km.labels_))

# The chosen model
kmeans = KMeans(n_clusters=5, init='k-means++', random_state=42, n_init=10)
y_kmeans = kmeans.fit_predict(X)        # cluster id per customer

print(kmeans.cluster_centers_)          # the 5 (income, spending) centres
print(np.bincount(y_kmeans))            # segment sizes: [81 39 22 35 23]

# The classic scatter — five colours, centroids starred
import matplotlib.pyplot as plt
for c in range(5):
    plt.scatter(X[y_kmeans == c, 0], X[y_kmeans == c, 1],
                s=60, label=f'Cluster {c+1}')
plt.scatter(kmeans.cluster_centers_[:, 0], kmeans.cluster_centers_[:, 1],
            s=300, c='yellow', marker='*', label='Centroids')
plt.title('Clusters of customers'); plt.xlabel('Income (k$)')
plt.ylabel('Spending Score'); plt.legend(); plt.show()

Listing 19.1 — The complete pipeline (source: k_means_clustering.py, with silhouette added).

💡

Pro Tip — the API detail: inertia_ is sklearn's WCSS; fit_predict returns labels; cluster_centers_ the centroids. Fixing random_state plus n_init=10 keeps segments reproducible across reruns — Chapter 3's habit, now for a non-deterministic learner.

7. Reading the Segments

The five real clusters, straight from the fitted model — each named by its position on the two axes, which is the entire art of segmentation naming:

Segment	Centre (income k$, spending)	n	Profile	Marketing action
Target	(87, 82)	39	High income, high spending	Loyalty perks, premium launches — protect at all costs
Careless	(26, 79)	22	Low income, high spending	Careful credit offers; impulse-friendly bundles
Careful	(88, 17)	35	High income, low spending	The prize: activation offers, quality messaging
Sensible	(26, 21)	23	Low income, low spending	Value lines; low-cost retention
Standard	(55, 50)	81	Middle of the mall	Volume campaigns; the segment that must not be offended

Two professional readings. The money is asymmetric: the Target segment is 39 customers (19.5%) but almost certainly a disproportionate share of margin, while the Careful 35 (17.5%) hold high income the mall hasn't converted — the two segments a CMO acts on first, with opposite plays. The Standard mass matters defensively: 81 customers (40%) with no extreme on either axis — generic campaigns targeting "everyone" are really campaigns targeting them, and the four named segments deserve better than the average of a blurred majority.

🌎

Real World — naming discipline: names like "Target" and "Careless" are hypotheses, not findings. The clusters are arithmetic; the interpretation (does "Careless" mean impulsive, or gift-shoppers?) needs the business in the room. A segment nobody can name is a segment that may not exist — validate against fresh data before printing the deck.

8. Strengths, Limits & the Road Ahead

Strengths	Limitations
Fast, scalable, easy to explain to anyone	k must be chosen in advance
Centroids give each segment a readable face	Spherical-cluster assumption — rings and moons carve badly
Elbow + silhouette usually converge on a defensible k	Init-lottery: unstable without k-means++ and restarts
The workhorse of customer segmentation everywhere	Outliers drag centroids; scaling usually required

K-Means demands k before it starts — and flatly refuses to tell you its own answer. The next chapter removes even that assumption: hierarchical clustering builds every k simultaneously — a tree of nested clusters whose dendrogram you cut wherever the data says cut — trading K-Means' speed for a picture that needs no k in advance and reveals structure at every scale.

🛠 Hands-On: Segment the MallIntermediate

Marketing accepted k = 5 but wants proof it isn't arbitrary, and a named segment table with sizes and actions. Rebuild the analysis end to end — elbow, silhouette, seeding check — and deliver the deck's one slide.

Steps

By hand: run K-Means on {2, 3, 4, 10, 11, 12} with k = 2 from seeds 2 and 12 — assignments, updates, final WCSS (expect 4).
Compute the WCSS table for k = 1..10; identify the elbow from the drop column, not the eyeball.
Compute silhouette scores for k = 2..10; verify the peak; check k = 6's verdict on the extra boundary.
Reproduce the seeding lottery: n_init = 1, ten seeds, random vs k-means++ — compare worst-case WCSS.
Fit k = 5 (n_init = 10); extract centres and sizes; name the segments with the two-axis discipline.
Write the one-slide recommendation: k, evidence (elbow + silhouette), the two priority segments, their actions.

View Solution / Walkthrough

import numpy as np, pandas as pd
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score

X = pd.read_csv('Mall_Customers.csv').iloc[:, [3, 4]].values

for k in range(1, 11):
    km = KMeans(n_clusters=k, init='k-means++', random_state=42,
                n_init=10).fit(X)
    s = silhouette_score(X, km.labels_) if k > 1 else float('nan')
    print('k=%2d WCSS=%9.1f sil=%s' % (k, km.inertia_,
          ('%.4f' % s) if k > 1 else '   —'))

for init in ['random', 'k-means++']:
    worst = max(KMeans(n_clusters=5, init=init, n_init=1,
                       random_state=s).fit(X).inertia_ for s in range(10))
    print('%-9s worst single-init WCSS: %.0f' % (init, worst))

km5 = KMeans(n_clusters=5, init='k-means++', random_state=42,
             n_init=10).fit(X)
for c in range(5):
    print(c, km5.cluster_centers_[c].round(0),
          (km5.labels_ == c).sum())

Expected results (real):

Step 1: {2,3,4} and {10,11,12}; centroids converge to 3 and 11 immediately; WCSS = 4.
Steps 2–3: drops 88,618 / 75,015 / 32,668 / 29,232 then 7,215 — elbow at 5; silhouette peaks 0.554 at k = 5 (k = 6 drops to 0.540).
Step 4: worst-case single-init WCSS: random 89,897 vs k-means++ 66,674 — the lottery is real, the smarter seeding halves it, n_init = 10 buys out of it.
Step 5: centres (87,82)·39 Target, (26,79)·22 Careless, (88,17)·35 Careful, (26,21)·23 Sensible, (55,50)·81 Standard.
Step 6: one slide — k = 5 by elbow + silhouette agreement; prioritise Target (protect) and Careful (activate); Standard handled with volume campaigns, never ignored.

🎯 Key Takeaways

Unsupervised means no judge: no labels, no confusion matrix — structure is validated by internal measures and business meaning.

K-Means = assign to nearest centroid, update to mean, repeat — by hand: {2,3,4,10,11,12} converges in one round to WCSS 4.

The elbow: WCSS drops 29k per cluster through k = 5, then 7k — and the silhouette peaks at the same k = 5 (0.554). Two instruments, one verdict.

Silhouette = (separation − cohesion)/max: near +1 snug, near 0 borderline, negative misassigned — the audit that catches rogue clusters.

Initialisation is a lottery: single random seeds ranged to 89,897 WCSS vs the 44,449 optimum — k-means++ halves it, n_init = 10 ends it.

Five segments, named on the axes: Target 39, Careless 22, Careful 35, Sensible 23, Standard 81 — names are hypotheses, actions are the deliverable.