Part V — Association Rule Learning

Chapter 21 · Apriori

Part V asks retail's oldest question: what goes into baskets together? No distances, no labels, no clusters — just co-occurrence, and three numbers that turn it into decisions: support, confidence and lift, every one computed by hand on a five-basket table before the machine touches 7,501 real transactions. Then the algorithm that makes the search survivable, and the nine real rules it found — led by a cheese most shoppers have never heard of.

📖 Chapter 21 of 33 ⏱ ~85 min read Mostly Core Stable 📝 Quiz · 30 MCQs

🎯 Learning Objectives

Compute support, confidence and lift by hand from a small basket table — and catch the traps each metric sets alone
Explain the combinatorial wall (2¹²⁰ itemsets) and how the Apriori principle prunes it
Choose min_support, min_confidence and min_lift with business rationales, not defaults
Read the real nine rules of the market-basket run and translate them into actions
Implement the pipeline with apyori: ragged transactions, rule extraction, the results DataFrame
Explain why the most popular item (mineral water) appears nowhere in the interesting rules

1. From Groups to Groceries

Two Parts asked "which customers group together"; this one asks "which items travel together". The unit of data is the transaction — one basket, an unordered set of products — and the deliverable is an association rule: people who buy X tend to also buy Y, written X → Y, scored by numbers a category manager can act on.

The running dataset: Market_Basket_Optimisation.csv — 7,501 baskets over one week of a grocery store, 20 item columns per row, ragged (a basket of 2 leaves 18 blank cells). 120 distinct products, and a familiar long tail: the most popular item, mineral water, appears in 1,788 baskets (support 0.238); eggs, spaghetti and french fries follow; by item #50 you are deep into single-digit-daily territory.

Item	mineral water	eggs	spaghetti	french fries	chocolate	green tea	milk	ground beef
Baskets	1,788	1,348	1,306	1,282	1,230	991	972	737
Support	0.238	0.180	0.174	0.171	0.164	0.132	0.130	0.098

📝

Note — remember mineral water's 0.238. One in four baskets contains it — a fact that will sabotage several naive rules in Section 7, and the reason lift exists.

2. Support, Confidence, Lift — by Hand

Five baskets, four products, every number by hand:

Basket	1	2	3	4	5
Items	bread, milk	bread, milk, eggs	bread, eggs	milk, eggs	bread, milk, eggs, cheese

Item counts: bread 4 (baskets 1,2,3,5), milk 3 (1,2,5), eggs 4 (2,3,4,5), cheese 1 (5). Now the three metrics on three candidate rules — chosen because together they commit every classic mistake:

Support(X → Y) = baskets with both X and Y / all baskets
Confidence(X → Y) = both / baskets with X
Lift(X → Y) = confidence / support(Y)

Support: how common the pairing is. Confidence: P(Y | X) — the conditional from Chapter 15. Lift: how much better than baseline (Y's raw popularity) the rule performs.

2.1 Rule 1: bread → milk (the plausible one)

supp(bread∧milk) = 3/5 = 0.60 · conf = 3/4 = 0.75 · lift = 0.75 / 0.60 = 1.25

Confidence looks strong — 75% of bread buyers take milk. But milk is in 60% of baskets anyway: the rule beats random by only 25%. Plausible ≠ interesting.

2.2 Rule 2: eggs → milk (the negative one)

supp(eggs∧milk) = 2/5 = 0.40 · conf = 2/4 = 0.50 · lift = 0.50 / 0.60 = 0.83

Lift below 1: egg buyers are less likely than average to buy milk. A real anti-association — the two products repel. Confidence alone would never tell you.

2.3 Rule 3: cheese → bread (the trap)

supp(cheese∧bread) = 1/5 = 0.20 · conf = 1/1 = 1.00 — perfect! · lift = 1.00 / 0.80 = 1.25

One basket. One co-occurrence. Confidence 100% — and it means exactly nothing: the sample is one event. The final trap: no metric may be read without its support (the sample size underneath).

⚠

Warning — the referee principle: support certifies the rule is worth counting, confidence measures the tendency, lift decides whether the tendency beats the baseline. A rule needs all three read together — high confidence on tiny support is noise; decent confidence with lift ≈ 1 is a popular-item illusion; lift < 1 is a repulsion, not a recommendation.

3. The Combinatorial Wall & the Apriori Principle

Why an algorithm at all — why not count everything? Because "everything" is 2¹²⁰ itemsets: every subset of the 120 products, a number with 36 digits. Even enumerating at a billion per second outlives the universe. The wall is real, and Apriori's principle (Agrawal & Srikant, 1994) is the door:

Every subset of a frequent itemset is frequent.

Contrapositive: if any subset is infrequent, the whole superset is dead on arrival. If {pasta, escalope} is frequent, then {pasta} and {escalope} must each be frequent — a basket containing both necessarily contains each. So: count single items first, keep only the frequent ones; build candidate pairs only from frequent singles; count those; build triples only from frequent pairs. The pruning is anti-monotone — support only shrinks as itemsets grow — and each level slaughters the search space before it is born.

Figure 21.1 — The level-wise search: each level's survivors are the only parents of the next level's candidates.

💡

Pro Tip — where the name comes from: "a priori" = known in advance. The principle lets you reject candidate itemsets before counting, on prior knowledge of their subsets — the algorithm's entire speed is foresight, not faster counting.

4. Choosing the Thresholds

The source's settings, each with its business rationale:

Parameter	Value	Why this value
`min_support = 0.003`	0.3% of baskets	≈ 22–23 baskets of 7,501 ≈ 3 sales per day over the week — the minimum a store would act on. Lower: noise and unicorn pairs; higher: only staples, no discoveries.
`min_confidence = 0.2`	20%	Deliberately low — confidence is the sieve's coarse screen; lift does the real filtering (Section 7 shows why high-confidence/low-lift rules are worthless).
`min_lift = 3`	3×	The referee: a rule must make the consequent three times more likely than baseline to count as a genuine association. Lift 1 = illusion; 3 = worth a meeting.
`min_length, max_length = 2`	pairs only	Rules of exactly two items: readable, placeable, bundlable. Triples (mushroom cream sauce → escalope and what else?) can wait for a bigger dataset.

🌎

Real World — the thresholds ARE the analysis: the same 7,501 baskets yield 3 throwaway rules at lift ≥ 5 and hundreds of popular-item illusions at lift ≥ 1.5. A data scientist who cannot say why min_support is 0.003 has not configured the analysis — they have copied it. Tie every threshold to a decision: what volume would we act on, what lift would justify moving a shelf?

5. The Real Rules

The run — 7,501 transactions, the Section 4 thresholds — returns exactly 9 rules, sorted here by lift (all real):

#	Rule (X → Y)	Support	Confidence	Lift	Reading
1	fromage blanc → honey	0.0033	0.245	5.16	Dessert pairing — niche but the strongest association in the store
2	light cream → chicken	0.0045	0.291	4.84	Cooking cluster: cream sauces
3	pasta → escalope	0.0059	0.373	4.70	The Italian dinner basket
4	pasta → shrimp	0.0051	0.322	4.51	Same cluster, second door
5	whole wheat pasta → olive oil	0.0080	0.271	4.12	The health-conscious Italian basket
6	tomato sauce → ground beef	0.0053	0.377	3.84	Bolognese, spotted by the algorithm
7	mushroom cream sauce → escalope	0.0057	0.301	3.79	Recipe-driven purchase
8	herb & pepper → ground beef	0.0160	0.323	3.29	The volume rule — 120 baskets/week, still lift 3.3
9	light cream → olive oil	0.0032	0.205	3.11	Crossing the two cooking clusters

Three readings before acting on any of them:

The rules are recipes. Pasta + escalope + shrimp; tomato sauce + ground beef; cream + chicken — the algorithm has rediscovered Italian home cooking from baskets alone. That is the signature of real association structure: rules that decode into human behaviour.
Lift and support trade off. The champion rule (lift 5.16) rides on 25 baskets — a boutique insight. Rule 8 (herb & pepper → ground beef) has five times the volume at still-strong lift: the rule a category manager would actually move shelves for.
Actions follow naturally: co-location (place fromage blanc by the honey), bundles (pasta + escalope meal deal), and recommenders ("with your ground beef…") — the three standard monetisations of association rules.

📋 Stable content — Reviewed: August 2026

6. Implementation in Python

Apriori ships not in sklearn but in apyori — a small third-party package (pip install apyori) that expects transactions as a list of lists of strings:

import pandas as pd

dataset = pd.read_csv('Market_Basket_Optimisation.csv', header=None)

# Ragged rows -> list of lists, blanks stripped (vital: 'nan' is not a product)
transactions = [[str(item) for item in row if pd.notna(item)]
                for row in dataset.values]
# 7,501 transactions; each an unordered basket

from apyori import apriori
rules = apriori(transactions=transactions,
                min_support=0.003, min_confidence=0.2,
                min_lift=3, min_length=2, max_length=2)
results = list(rules)                     # 9 rules

# Unpack the nested result objects into a readable DataFrame
def inspect(results):
    lhs         = [tuple(r[2][0][0])[0] for r in results]
    rhs         = [tuple(r[2][0][1])[0] for r in results]
    supports    = [r[1]              for r in results]
    confidences = [r[2][0][2]        for r in results]
    lifts       = [r[2][0][3]        for r in results]
    return list(zip(lhs, rhs, supports, confidences, lifts))

rules_df = pd.DataFrame(inspect(results),
                        columns=['LHS', 'RHS', 'Support',
                                 'Confidence', 'Lift'])
print(rules_df.sort_values('Lift', ascending=False))

Listing 21.1 — The complete pipeline (source: apriori.py, blank-stripping made explicit).

💡

Pro Tip — the classic bug: the source converts every cell with str(), blanks included, so the string 'nan' becomes the store's 121st "product" — with suspiciously high frequency. Strip missing cells before building transactions (as above), or audit transactions[0] for the ghost item before trusting any rule.

7. Confidence's Trap, Lift's Verdict

One puzzle remains from Section 1: mineral water — 1,788 baskets, the most popular product in the store — appears in none of the nine rules. Not as antecedent, not as consequent. Why did the star of the dataset vanish?

Because lift refuses to be impressed by popularity. For "X → mineral water" to clear lift 3, X-buyers would need to buy water at 3 × 0.238 ≈ 71% — and nothing in a grocery basket predicts water that strongly, because everyone buys water nearly independently of everything else. Water pairs with pasta at roughly its baseline rate; the naive rule "pasta → water" would show healthy confidence (near 0.24+) and lift ≈ 1: the bread → milk illusion of Section 2, at scale. The min_lift = 3 gate quietly deleted every such rule — which is precisely its job.

Rule shape	Confidence	Lift	Verdict
fromage blanc → honey (real #1)	0.245 — modest	5.16	Genuine association — honey buyers' rare partner
any-X → mineral water (hypothetical)	≈ 0.24+ — looks fine	≈ 1.0	Popular-item illusion — deleted by the lift gate
eggs → milk (hand table)	0.50 — even better	0.83	Anti-association — the pair repels

The pattern generalises into this Part's central discipline: confidence is P(Y|X); lift is P(Y|X) ÷ P(Y). Only the division removes the popularity subsidy. Every rule you ever present should carry its lift — and its support, the sample size that makes both meaningful.

8. Strengths, Limits & the Road Ahead

Strengths	Limitations
Simple, interpretable, directly actionable rules	Combinatorial cost even after pruning — long itemsets explode
Unsupervised — needs no labels, no distances	Thresholds are business choices with no automatic setting
Lift separates real associations from popular-item illusions	Pair-only view here; multi-item baskets need patience or Eclat
A mature, battle-tested algorithm (since 1994)	Rules are correlations — a bundle hint, never a causal claim

The next chapter keeps the data and the support counting, throws away confidence and lift, and asks a leaner question: not "what implies what" but "which itemsets are frequent, full stop". Eclat's vertical transaction-id layout makes that question brutally fast — and the comparison between the two philosophies is Part V's closing lesson.

🛠 Hands-On: Audit the Nine RulesIntermediate

A category manager asks for the rules behind a shelf reshuffle — and wants the traps demonstrated before the meeting. Produce the hand audit, the real run, and the one-slide recommendation.

Steps

By hand: from the five-basket table, compute support/confidence/lift for milk → bread and bread → eggs; explain why direction changes the numbers.
By hand: show cheese → eggs has confidence 1.00; state the two reasons it is worthless.
Run Apriori with the source thresholds; verify 9 rules; sort by lift and by support.
Relax min_lift to 1.0; count the rules and find the first mineral-water illusion; compute its lift by hand from the output.
Identify the volume rule (herb & pepper → ground beef) and convert support 0.0160 into baskets per week.
Write the one-slide recommendation: the two rules to act on (one boutique, one volume), the actions, and the causal caveat.

View Solution / Walkthrough

import pandas as pd
from apyori import apriori

dataset = pd.read_csv('Market_Basket_Optimisation.csv', header=None)
transactions = [[str(v) for v in row if pd.notna(v)] for row in dataset.values]

def run(min_support, min_conf, min_lift):
    rs = list(apriori(transactions=transactions,
                      min_support=min_support, min_confidence=min_conf,
                      min_lift=min_lift, min_length=2, max_length=2))
    rows = [(tuple(r[2][0][0])[0], tuple(r[2][0][1])[0],
             r[1], r[2][0][2], r[2][0][3]) for r in rs]
    return pd.DataFrame(rows, columns=['LHS','RHS','Supp','Conf','Lift'])

df = run(0.003, 0.2, 3)
print(len(df), 'rules'); print(df.sort_values('Lift', ascending=False).head())

loose = run(0.003, 0.2, 1.0)
print(len(loose), 'rules at lift>=1')
print(loose[loose.RHS == 'mineral water'].head())

Expected results (real):

Step 1: milk → bread: supp 3/5 = 0.60, conf 3/3 = 1.00, lift = 1.00/0.80 = 1.25; bread → eggs: supp 3/5, conf 3/4 = 0.75, lift = 0.75/0.80 = 0.94. Direction matters: X and Y swap the denominator.
Step 2: cheese → eggs: conf 1/1 = 1.00 — worthless because (a) support 1/5 = 0.20 is a single basket (sample of one), and (b) lift = 1.00/0.80 = 1.25 barely beats baseline.
Step 3: 9 rules; lift-sorted top = fromage blanc → honey (5.16); support-sorted top = herb & pepper → ground beef (0.0160).
Step 4: at lift ≥ 1 the rule count explodes; the water rules enter with lift ≈ 1 — e.g. some-X → mineral water at conf ≈ 0.24–0.35 and lift ≈ 1.0–1.5: the popular-item illusion, now visible in the output.
Step 5: 0.0160 × 7,501 ≈ 120 baskets per week (~17/day) — the volume that justifies moving shelf positions.
Step 6: act on fromage blanc → honey (co-placement; boutique margin) and herb & pepper → ground beef (bundle + recommender; volume) — with the caveat that association is correlation: test the intervention, don't assume the causal arrow.

🎯 Key Takeaways

Support = frequency worth counting, confidence = P(Y|X), lift = confidence ÷ P(Y) — the referee that strips out an item's popularity.

The hand table's three traps: conf 0.75/lift 1.25 (popular-item illusion), conf 0.50/lift 0.83 (repulsion), conf 1.00 on one basket (noise).

Apriori's principle — subsets of frequent sets are frequent — prunes the 2¹²⁰ wall level by level; the speed is foresight, not faster counting.

Thresholds are business decisions: support 0.003 = 3 sales/day; lift 3 = triple the baseline before a rule earns a meeting.

Nine real rules, and they are recipes: pasta+escalope, cream+chicken, tomato sauce+ground beef — behaviour decoded from baskets alone.

Mineral water — 24% of baskets — appears in no rule: nothing predicts it 3× better than baseline. That absence is lift doing its job.

🎯 Learning Objectives

1. From Groups to Groceries

2. Support, Confidence, Lift — by Hand

2.1 Rule 1: bread → milk (the plausible one)

2.2 Rule 2: eggs → milk (the negative one)

2.3 Rule 3: cheese → bread (the trap)

3. The Combinatorial Wall & the Apriori Principle

4. Choosing the Thresholds

5. The Real Rules

6. Implementation in Python

7. Confidence's Trap, Lift's Verdict

8. Strengths, Limits & the Road Ahead

🛠 Hands-On: Audit the Nine RulesIntermediate

Steps

🎯 Key Takeaways

Test Your Understanding