Chapter 22 · Eclat

1. The Leaner Question

Chapter 21 produced rules: directional, lift-refereed statements — fromage blanc → honey — with three metrics to read together. Eclat asks for less and gets it faster: which itemsets (unordered sets of items) clear a minimum support? No arrow, no confidence, no lift. {mineral water, spaghetti} is frequent — full stop — with no claim about which causes which or whether the pairing beats baseline.

What does the leaner question cost and buy? It costs insight machinery: from frequent itemsets you can always compute Chapter 21's rules afterwards (confidence and lift are simple ratios over supports), but Eclat itself ranks nothing and referees nothing — its raw output, sorted by support, systematically promotes popular items. It buys speed and simplicity: fewer metrics, one counting operation, and — as Section 6 measures — a substantially faster search, which is exactly what you want when the itemset length grows or the basket count explodes.

2. The Vertical Layout, by Hand

Apriori scanned baskets (rows) to count itemsets. Eclat flips the table sideways — the vertical (TID) layout: for each item, the list of transaction IDs containing it. Chapter 21's five baskets, transposed:

And now the entire algorithm is one operation — set intersection:

3. Eclat's Search & the Source's Confession

The search inherits Apriori's pruning (a pair whose singles are infrequent is never counted — anti-monotonicity is layout-independent) but organises work differently: depth-first through prefix groups. Start from mineral water's TID list; intersect with each co-item; each surviving pair becomes a new prefix branch for triples; recurse. Apriori instead sweeps levels breadth-first, re-reading the database; Eclat touches the database once — everything after is intersection arithmetic.

One honest confession about the source code: eclat.py actually calls apyori again — Apriori's machinery — simply discarding the confidence and lift columns and ranking by support alone. That reproduces Eclat's question (support-only output) without Eclat's engine. The modern, honest implementation of the vertical family is mlxtend's fpgrowth (FP-growth: a compressed prefix-tree over the transactions — a cousin of Eclat's prefix idea), which this chapter runs for its real numbers and its timing.

4. The Real Itemsets

FP-growth, min_support = 0.004 (≈ 30 baskets ≈ 4+ per day), itemsets of up to three items, on the real 7,501 baskets:

The top of each size class, by support (all real):

Three readings, each a Part V lesson in miniature:

• Every top pair contains mineral water. The #1 pair (water + spaghetti, 0.0597 ≈ 448 baskets/week) is genuinely massive volume — but it is volume, not discovery: water rides along with everything (Chapter 21's 0.238 support doing exactly what we warned). Support-only rankings are popularity contests with extra steps.
• The interesting itemsets are invisible here. Chapter 21's insight champion — fromage blanc + honey, lift 5.16 — has support 0.0033, below even this chapter's 0.004 bar. The two metrics select different champions: support finds volume, lift finds association. A store needs both reports.
• Triples tell the water story again, deeper. All six top triples are {mineral water, spaghetti, X} or {mineral water, chocolate/milk/eggs} — the staples' co-purchase core. Fine for inventory and placement of high-traffic goods; useless for the cross-sell discovery a lift-based analysis provides.

6. Speed: the Real Motivation

Same baskets, same min_support = 0.003, same one-hot matrix — timed (real):

A 3.5× gap on a toy-sized dataset, and the gap widens with data: Apriori's cost scales with candidate counts per level (each level re-derives supports from scratch), while the vertical family reuses every intersection's result downward through the prefix tree — support of {water, spaghetti, ground beef} is computed from {water, spaghetti}'s list intersected with ground beef's, never from the raw database. On million-basket retail data this is the difference between an interactive tool and an overnight job.

7. Apriori vs Eclat — Settled

The professional's summary, one sentence each: Apriori answers "why would a customer buy Y" (with lift as the lie detector); Eclat answers "what moves together in bulk" (with speed as the enabler). Same baskets, complementary reports — and the rule mining journey of this course ends with both on the table.

8. Part V Retrospective & the Road Ahead

Two chapters, one dataset, two philosophies of co-occurrence: rules with a referee (fromage blanc → honey, lift 5.16 — discovery) and itemsets with speed (water + spaghetti, support 0.0597 — volume). Together they close the unsupervised arc that Part IV opened: clustering asked who groups, association asked what groups, and neither needed a single label.

Part VI changes the shape of time itself. So far every dataset was static — a table to be studied. Reinforcement learning makes data arrive as consequences of actions: an ad shown, a reward observed, a strategy updated — the exploration-versus-exploitation dilemma, the multi-armed bandit, and Upper Confidence Bound, where an algorithm must gamble wisely rather than analyse quietly.