Part 5 — Failure Analysis and System Improvement

Semantic boundary failures

Where failures live·Evaluation·6 min read

The hardest failures cluster where two correct answers sit a hair apart in meaning — a ranking versus a breakdown, an inclusive versus exclusive filter. These semantic boundary failures are where valid-but-wrong lives, and understanding them is the deepest layer of failure analysis.

Confusion analysis showed that the system swaps one category for an adjacent one. This chapter is about why those particular categories are so easy to swap: they sit on either side of a semantic boundary — a line where the meaning of the correct answer changes on a subtle cue. These are the failures that produce perfectly valid but wrong outputs, the hardest ones to catch and the most instructive ones to study, because each names a distinction the system genuinely hasn't grasped.

What you will understand by the end

  • What a semantic boundary is and why failures concentrate there.
  • Why boundary failures are exactly the valid-but-wrong outputs from Part 4.
  • Why these are the hardest failures to detect and the most valuable to fix.
  • How to turn a boundary failure into a precise fix and a regression test.

Where meaning turns on a small cue

A semantic boundary is a point in the input space where a small change in wording flips the correct answer from one thing to a closely related other. "Top regions by revenue" and "revenue by region" differ by a few words, but one wants a ranking and the other a breakdown — entirely different queries. The two intents are neighbours in meaning-space, separated by a cue the system has to notice and interpret correctly. Failures pile up here because the categories are genuinely close and the deciding signal is subtle.

Key idea

Failures aren't spread evenly across meaning-space; they concentrate at boundaries where two correct answers sit close together and a small cue decides between them. The system handles the clear middle of each category and slips at the edges — which is why a boundary failure is a precise statement about the exact distinction it hasn't learned.

Boundary failures are the valid-but-wrong failures

This is where two threads of the book meet. A semantic boundary failure produces an output that is structurally flawless — right schema, right fields, legal values — and semantically wrong, because it landed on the wrong side of the boundary. The valid ≠ correct gap is the space between adjacent meanings, and boundary failures live in it. No validity check can catch them, because nothing about the shape is wrong; only a check against what the answer should mean does.

Observed evidence

This project's headline finding is a semantic boundary failure. For "Revenue by region last quarter" the raw model returned a valid top_n_by_metric intent — a ranking — when the boundary called for metric_by_dimension_with_filter, a breakdown. Perfect JSON, wrong meaning, off by exactly one semantic step. It's why 100% schema-validity coexisted with 45.5% correctness: the failures weren't malformed, they were on the wrong side of a boundary. Read the structured-output-semantics analysis →

Hardest to detect, most valuable to fix

Boundary failures are the hardest class because every cheap signal says they're fine — they parse, they validate, they look plausible, and a lenient semantic scorer can even mark them correct. Catching them needs strict, meaning-level scoring and, often, the adversarial cases built to probe the boundary directly. But they are also the most valuable to study: each one names a specific, learnable distinction, so fixing it is targeted — a prompt clarification that spells out the cue, examples that contrast the two sides, or a correction rule keyed to the boundary. Fix the boundary and you often fix a whole cluster of failures at once.

Mental model

Semantic boundaries are lines in meaning-space where a subtle cue flips the correct answer between two adjacent categories. Failures concentrate there, they're structurally valid and semantically wrong (the valid-but-wrong gap made concrete), they evade every cheap check, and each one names an exact distinction to teach — so fixing a boundary clears a cluster of failures.

Common mistakes

  • Assuming valid-and-plausible means correct. Boundary failures pass every surface check; only meaning-level scoring catches them.
  • Treating them as random errors. They're concentrated at specific distinctions the system hasn't learned, not noise.
  • Scoring boundaries with a lenient semantic metric. Its false positives pass wrong-side-of-the-boundary answers; use strict, canonical scoring here.
  • Fixing one case at a time. A boundary failure represents a whole cluster; fix the distinction, not the single instance.

Practical guidance

  • Identify the boundaries in your task — the pairs of adjacent correct answers separated by a subtle cue — and probe them with adversarial cases.
  • Score boundaries with strict, meaning-level rules (exact match on a canonical form); don't let a lenient scorer wave them through.
  • Fix at the distinction level — clarify the cue in the prompt, add contrasting examples, or a boundary-keyed correction rule — so one fix clears the cluster.
  • Turn each fixed boundary into a regression test so the distinction stays learned.

Summary

  • Failures concentrate at semantic boundaries, where a subtle cue flips the correct answer between adjacent categories.
  • Boundary failures are the valid-but-wrong outputs — structurally perfect, semantically off by one step — so validity checks can't catch them.
  • They're the hardest to detect (every cheap signal says fine) and the most valuable to fix (each names a learnable distinction).
  • This project's ranking-vs-breakdown error is the canonical example — the reason 100% valid sat on 45.5% correct.

Knowledge check

A failure produces schema-valid JSON that a semantic-similarity scorer marks "correct," but a human says it's wrong. What kind of failure is this, and what scoring does it demand?

A semantic boundary failure: the output landed on the wrong side of a subtle distinction, so it's structurally valid and near the right answer in meaning-space (which is why the lenient similarity scorer passed it) but actually wrong. It demands strict, meaning-level scoring — exact match against a canonical reference — because the whole point is that surface validity and mere similarity can't distinguish the two sides of the boundary; only a strict semantic check can.

Why is fixing a boundary failure often high-leverage rather than a one-off patch?

Because a boundary failure isn't one weird case — it's an instance of a whole cluster of inputs that all sit near the same distinction the system hasn't learned. Fixing the distinction itself (clarifying the deciding cue in the prompt, adding contrasting examples, or a boundary-keyed correction rule) resolves the entire cluster at once, not just the single trace you saw. That's why naming the boundary — as this project did with ranking-vs-breakdown — yields a targeted fix worth many points.

Related chapters