Part 5 — Failure Analysis and System Improvement

Root-cause analysis

From failure to fix·Evaluation·7 min read

Failure analysis ends where it should: turning a characterised failure into a durable fix. Root-cause analysis is the method — reproduce, localise the stage, characterise the pattern, hypothesise the cause, fix, and verify no regression — plus the habit that compounds, turning every failure into a permanent test case.

Everything in Part 6 has been building to this: you can open traces, sort failures into types, see the confusions and boundaries, and tell structural from semantic. Root-cause analysis ties it into a repeatable method for going from "the system failed here" to "the system won't fail here again." The discipline is to find the actual cause — not the first plausible one — fix it, verify the fix didn't break anything else, and capture the failure so it can never silently return.

What you will understand by the end

  • The root-cause loop: reproduce → localise → characterise → hypothesise → fix → verify.
  • Why you chase the true cause, not the first plausible-looking one.
  • Why every fix must be checked for regressions before it counts.
  • The compounding habit: turning each root-caused failure into a permanent test case.

The root-cause loop

A disciplined pass on a failure (or a whole failure type) runs the same steps:

  1. Reproduce. Make the failure happen reliably — same input, same system version. A failure you can't reproduce, you can't diagnose or verify a fix against.
  2. Localise. Use the trace's stages to find where it broke — retrieval, model, parse, validation, correction, execution. Is it structural or semantic?
  3. Characterise. Find the pattern across cases — which type, which confusion, which boundary. One case is an anecdote; the pattern is the target.
  4. Hypothesise the cause. Ask why the pattern happens — a missing prompt instruction, an under-constrained schema, a retrieval gap, a genuinely hard distinction — and state it as a testable claim.
  5. Fix. Apply the fix that matches the cause and the boundary (prompt, schema, guardrail, retriever, data), the smallest change that addresses the cause, not the symptom.
  6. Verify. Re-run and confirm the targeted failures are fixed and nothing else regressed.
Key idea

Root cause is the difference between treating a symptom and curing a cause. "The model output the wrong pattern" is a symptom; "the prompt never distinguishes ranking language from grouping language, so the model guesses at the boundary" is a cause. You fix causes — symptom fixes paper over the failure and it resurfaces in a new disguise.

Chase the true cause, not the first plausible one

The most common root-cause error is stopping at the first explanation that sounds right. A wrong answer looks like a weak model, so you reach for a bigger model — when the real cause was a retrieval miss that no model size fixes, or a prompt that never stated the rule. The stage-localisation step is what protects you: it forces you to find where the failure entered before naming why, so you don't fix the wrong component. "The model was wrong" is a hypothesis to test against the trace, not a conclusion.

Watch out

Reflexively blaming "the model" is how effort gets wasted. An empty answer might be a retrieval miss; a broken response a parse bug; a wrong query an under-constrained schema. Each has a different, cheaper fix than "get a better model." Localise to the stage first; the true cause is often upstream of, and far cheaper than, the model.

Verify — a fix isn't a fix until it's checked

A change that fixes the targeted failures but breaks others is not an improvement, as guardrails made vivid. So verification is part of the loop, not an afterthought: re-run the full evaluation, confirm the targeted type improved, and confirm the aggregate and the distribution didn't regress elsewhere. Only a change that is net-positive and regression-checked counts as a fix.

Observed evidence

This project is a worked root-cause analysis end to end: the failure (low 3B accuracy) was reproduced, localised to the model stage, characterised as the ranking-vs-breakdown boundary confusion, hypothesised to come from the model missing a subtle cue, fixed with a targeted correction rule (+25 points), and — crucially — verified, which is how the 7B regression was caught rather than shipped. The whole method, on real data. Read the analysis end to end →

The habit that compounds: failures become tests

The final step turns analysis into a system that improves over time: every root-caused failure becomes a permanent test case. The specific input that failed, with its correct answer, goes into the evaluation set as a regression test, so that exact failure — and its whole cluster — can never silently return. Do this consistently and your evaluation set grows precisely along the dimensions your system actually fails on, which is the loop continuous evaluation is built around.

Mental model

Root cause = reproduce → localise the stage → characterise the pattern → hypothesise the true cause → fix the cause → verify no regression. Chase the real cause (often upstream of the model), not the first plausible one; a fix isn't a fix until it's regression-checked; and every root-caused failure becomes a permanent test so it can't come back.

Common mistakes

  • Stopping at the first plausible cause. "The model was wrong" is a hypothesis; localise to the stage before concluding.
  • Fixing the symptom. Patching the output instead of the cause makes the failure resurface in a new form.
  • Skipping verification. An unchecked fix can regress other cases; net-positive and regression-checked or it doesn't count.
  • Not capturing the failure. A root-caused failure that never becomes a test can silently return.

Practical guidance

  • Run the loop on failure types, not just single cases — reproduce, localise, characterise, hypothesise, fix, verify.
  • Localise before you blame: find the stage the failure entered; the cause is often upstream of (and cheaper than) the model.
  • Verify every fix against the full evaluation — targeted type improved, aggregate and distribution not regressed.
  • Turn every root-caused failure into a permanent test case so it, and its cluster, can't silently recur.

Summary

  • Root-cause analysis is a loop: reproduce → localise → characterise → hypothesise → fix → verify.
  • Chase the true cause (often upstream of the model), not the first plausible one — localise the stage first.
  • A fix isn't a fix until it's regression-checked: net-positive across the whole evaluation.
  • Turn every root-caused failure into a permanent test case — the compounding habit that ends Part 6 and opens continuous evaluation.

Knowledge check

Users report empty answers. An engineer says "the model's too weak, let's upgrade it." Walk through why root-cause discipline would pause that.

Because "the model is too weak" is the first plausible cause, not a localised one. The discipline says reproduce the empty answer, then localise using the trace's stages — and an empty answer is very often a retrieval miss (nothing relevant fetched, so the model had nothing to work with) or a parse/validation drop, neither of which a bigger model fixes. Only after localising to the stage that actually failed do you hypothesise a cause and fix it — likely far cheaper and more effective than a model upgrade aimed at the wrong stage.

You've fixed the targeted failure type and the aggregate went up 4 points. Why isn't the root-cause loop finished?

Because you haven't verified no regression. A rise in the aggregate can coexist with breaking other cases (the mean hides a lopsided change), as guardrails showed. You must check the distribution of what changed — confirm the targeted type improved and nothing critical regressed elsewhere — before the fix counts. And the loop only truly closes when you capture the failure as a permanent test case, so it and its cluster can't silently return.

Related chapters