Part 5 — Failure Analysis and System Improvement

Confusion analysis

Classifying failures·Evaluation·6 min read

When a task has categories, its failures aren't random — they're specific confusions of one category for another. A confusion matrix makes those directional mistakes visible, so you can see that the system reliably reads 'breakdown' as 'ranking' and fix the exact distinction it hasn't learned.

For any task with a taxonomy of categories — classes, patterns, labels — failures have a direction: the system didn't just get it wrong, it predicted category B when the answer was category A. Which categories get confused for which is not random, and the tool that exposes the structure is the confusion matrix. It turns "the classifier is 80% accurate" into "it reliably mistakes this for that" — the difference between a number and a diagnosis.

What you will understand by the end

  • What a confusion matrix is and what its structure reveals.
  • Why confusions are directional and usually systematic, not random.
  • How to read a confusion pattern into a specific, fixable distinction.
  • Where confusion analysis fits between error taxonomies and boundary failures.

The confusion matrix

Lay predicted categories against actual categories in a grid; each cell counts how often the system predicted the column's category when the truth was the row's. The diagonal is correct predictions; every off-diagonal cell is a specific confusion.

                    predicted →
                 ranking   breakdown   filter
   actual ↓
   ranking     [   38   ]     2          0        ← mostly right
   breakdown   [   14   ]    22          1        ← 14 breakdowns misread as ranking!
   filter          1          0         19
                    ▲
             a big off-diagonal cell = a systematic confusion to fix

The value isn't the accuracy on the diagonal — it's the shape of the off-diagonal. A big cell tells you exactly which pair of categories the system can't tell apart, and in which direction.

Key idea

Failures on a categorical task are directional and systematic: the system doesn't fail randomly, it confuses specific categories for specific others. The confusion matrix makes that structure visible, converting an aggregate accuracy into a precise statement — "breakdowns are being read as rankings" — that names the exact distinction to fix.

Direction matters

"A confused with B" and "B confused with A" are different bugs with different fixes and different costs. A system that over-predicts a safe default category fails differently from one that over-predicts a dangerous action. The confusion matrix keeps the direction — the cell above the diagonal is not the cell below it — so you fix the actual asymmetric mistake, not a symmetric average of it. This is also why a single accuracy number is too coarse: it sums the whole off-diagonal into one figure and erases the direction entirely.

Watch out

A modest overall accuracy can hide a single dominant confusion: the system is near-perfect everywhere except one pair of categories it systematically swaps. That concentrated confusion is often one fixable distinction, not diffuse weakness — and you'd never see it in the aggregate. Always look at the off-diagonal structure before concluding the model is "generally weak."

From confusion to fix

A dominant off-diagonal cell is a lead. It says: the system has not learned the distinction between these two categories, in this direction. That points straight at a fix — clarify the distinction in the prompt, add training or few-shot examples that separate the pair, or add a correction rule that catches the specific swap. And because the confused pair are usually semantically adjacent categories, confusion analysis flows directly into semantic boundary failures, where the why of the confusion lives.

Observed evidence

This project's signature failure is a confusion: raw models predict top_n_by_metric (ranking) when the truth is metric_by_dimension_with_filter (breakdown), overwhelmingly in that direction — a big off-diagonal cell for "by region"-style phrasings. Naming the confusion, with its direction, is what let the correction policy target precisely that swap and add 25 points. A confusion matrix turns 45.5% into "fix the ranking-for-breakdown swap." See the dominant confusion in the trials →

Mental model

On a categorical task, failures confuse specific categories for specific others, directionally. A confusion matrix shows which pairs and in which direction, turning an accuracy number into a named, fixable distinction. Look at the off-diagonal shape: a dominant cell is usually one fixable confusion, not general weakness — and its direction tells you which mistake to fix.

Common mistakes

  • Stopping at overall accuracy. It sums the off-diagonal into one number and erases which categories are confused.
  • Ignoring direction. "A→B" and "B→A" are different bugs; a symmetric summary hides the actual asymmetric mistake.
  • Reading a dominant confusion as general weakness. It's often one distinction the system hasn't learned, not diffuse poor performance.
  • No confusion view on categorical tasks. If the task has categories, the matrix is the natural diagnostic — not having it wastes structure.

Practical guidance

  • Build a confusion matrix for any categorical task and read the off-diagonal, not just the diagonal accuracy.
  • Attack dominant confusions first, respecting direction — fix the specific swap, weighted by how costly that direction is.
  • Translate each confusion into a targeted fix: prompt clarification, separating examples, or a correction rule for the exact pair.
  • Hand the confused pair to semantic boundary analysis to understand why they're adjacent.

Summary

  • Failures on categorical tasks are directional and systematic confusions of one category for another.
  • A confusion matrix exposes which pairs and in which direction — turning accuracy into a named, fixable distinction.
  • A dominant off-diagonal cell is usually one fixable confusion, invisible in the aggregate.
  • This project's ranking-for-breakdown swap is exactly such a confusion — naming it made the fix precise.

Knowledge check

A classifier is 82% accurate. The confusion matrix shows it's near-perfect except that class C is predicted 30% of the time when the truth is class D. What does this tell you that the 82% didn't?

That the weakness is not general — the system is strong everywhere except one specific, directional confusion: it reads D as C. That's likely a single learnable distinction (D and C are probably semantically adjacent), fixable with a targeted prompt clarification, separating examples, or a correction rule for the D→C swap. The 82% aggregate summed that concentrated confusion into diffuse-looking weakness and hid both which pair and which direction — the two things you need to fix it.

Why does the direction of a confusion matter for the fix?

Because "A predicted when truth is B" and "B predicted when truth is A" are different mistakes with different mechanisms and different costs — one might be safe (over-predicting a benign default), the other harmful. A fix (prompt cue, example, correction rule) targets a specific direction, and the costs are asymmetric, so collapsing the two into a symmetric "A and B get confused" would aim the fix wrong and misjudge the harm. The matrix preserves direction so you fix the actual mistake.

Related chapters