Part 9 — Continuous Evaluation

LLM-as-judge

Scaling judgment·Evaluation·6 min read

Open-ended outputs can't be exact-matched and humans don't scale — so you use an LLM to score them. This chapter is how LLM-as-judge works, why it's so attractive, and the sharp caveat that runs through the next chapter: a judge is itself an unvalidated LLM system, with its own biases, until you prove otherwise.

Some tasks have no single right answer — a summary, an explanation, a free-form response — so exact match is useless and you need judgment. Human judgment is the gold standard but doesn't scale. The increasingly standard answer is LLM-as-judge: use a language model to score outputs against a rubric, at the speed and cost of an API call. It's genuinely powerful and genuinely dangerous — a judge is itself an LLM system that can be biased and wrong, and treating its scores as ground truth without validation is one of the easier ways to fool yourself.

What you will understand by the end

  • What LLM-as-judge is and why it's so attractive for open-ended tasks.
  • The common forms: scoring against a rubric, and pairwise preference.
  • Why an LLM judge is itself an unvalidated system with characteristic biases.
  • Why this chapter hands directly to judge calibration.

Judgment at the speed of an API call

LLM-as-judge uses a strong model, given a rubric, to score another model's output. The two common shapes:

  • Rubric / pointwise scoring. Give the judge the input, the output, and a set of criteria ("is it faithful to the source? does it answer the question? is the format right?") and ask for a score or a per-criterion verdict — a rubric method with the model as grader.
  • Pairwise preference. Show the judge two outputs and ask which is better. Often more reliable than absolute scores, because relative judgments are easier than calibrated numbers — and it maps neatly onto A/B comparisons.

The appeal is obvious: judgment that used to need a human panel now runs on every output, every change, cheaply — making continuous evaluation of subjective quality actually feasible.

Key idea

LLM-as-judge makes subjective quality measurable at scale — the one thing that lets you run continuous evaluation on open-ended tasks where exact match can't reach and humans can't keep up. It's the workhorse that extends the whole eval program to summaries, explanations, and free-form answers.

A judge is an LLM system — with all that implies

Here's the caveat you must carry: an LLM judge is a language model doing a task, so everything this book said about LLM systems applies to it. It can be confidently wrong, it's sensitive to its prompt, and it has documented biases:

  • Position bias — in pairwise mode, favouring the first (or second) option regardless of quality.
  • Verbosity bias — rating longer, more elaborate answers higher even when they're not better.
  • Self-preference — a judge model tending to favour outputs from its own family or style.
  • Being fooled by fluency — a confident, well-written wrong answer can score higher than a hesitant right one (the valid-but-wrong trap, now in the judge).
Watch out

An LLM judge's score is not ground truth — it's the output of an unvalidated system that may be systematically biased. Building a release gate on a raw, uncalibrated judge means gating on a number you haven't verified against reality; a verbosity-biased judge will happily promote changes that just make outputs longer. Never treat a judge as an oracle before you've calibrated it.

Get more from a judge, carefully

Some practices make judges more reliable — but none remove the need to validate them:

  • Prefer pairwise over absolute scoring where you can; relative judgments are more stable.
  • Give a clear, specific rubric rather than "rate 1–10"; vague criteria maximise bias.
  • Swap positions and average to cancel position bias; ask for reasoning before the verdict.
  • Use a strong judge model, and keep the judge's prompt versioned like any other part of the system.

These help, but a judge that hasn't been checked against human labels is still a guess. That check is the whole subject of the next chapter.

Observed evidence

This project's task was designed to have a single canonical answer, so it uses deterministic exact match and needs no judge at all — which is itself the lesson: when you can make the task exact-matchable, do, and skip the judge's biases entirely. LLM-as-judge is the tool you reach for only when the output is genuinely open-ended and a canonical answer isn't possible. Why exact match beat a fuzzy judge here →

Mental model

**LLM-as-judge scores open-ended outputs against a rubric (or by pairwise preference) at API speed, making subjective quality measurable at scale. But the judge is an unvalidated LLM system with real biases — position, verbosity, self-preference, fluency — so its score is a hypothesis, not ground truth, until calibrated. Prefer exact match when the task allows; reach for a judge only when it doesn't.

Common mistakes

  • Treating judge scores as ground truth. They're outputs of an unvalidated, possibly biased system — calibrate before you trust.
  • Using a judge where exact match would work. If the task has a canonical answer, deterministic scoring avoids all the judge's biases.
  • Vague rubrics. "Rate 1–10" with no criteria maximises bias and variance; specify what good means.
  • Ignoring position/verbosity bias. Un-mitigated, a judge can reward order or length instead of quality.

Practical guidance

  • Reach for a judge only for genuinely open-ended outputs; use exact match whenever a canonical answer exists.
  • Prefer pairwise judgments, give a specific rubric, swap positions and average, and ask for reasoning before the verdict.
  • Version the judge model and its prompt as part of the evaluation configuration.
  • Never gate on a raw judge — calibrate it against human labels first.

Summary

  • LLM-as-judge scores open-ended outputs against a rubric or by pairwise preference, making subjective quality measurable at scale.
  • It's the workhorse for tasks exact match can't reach and humans can't keep up with.
  • But the judge is an unvalidated LLM system with position, verbosity, self-preference, and fluency biases — its score is not ground truth.
  • Use exact match when you can; when you can't, use a judge — and calibrate it before trusting it.

Knowledge check

A team uses an LLM judge to gate releases, and over months their outputs get steadily longer with no real quality gain. What likely happened?

Verbosity bias: the judge systematically rates longer, more elaborate answers higher regardless of whether they're actually better, so every release that made outputs longer "won" the gate and got promoted. Because they gated on a raw, uncalibrated judge, they optimised the judge's bias instead of real quality. The fix is to calibrate the judge against human labels (and check for length bias specifically), mitigate it (length-controlled prompts, pairwise with position swaps), and not treat the judge's score as ground truth in the gate.

Your task can be designed so each input has one canonical correct answer. Should you use an LLM judge? Why or why not?

No — use deterministic exact match on a normalized form. When a canonical answer exists, exact match is cheaper, perfectly reproducible, and can't be fooled by an LLM judge's position, verbosity, self-preference, or fluency biases. An LLM judge is the tool for genuinely open-ended outputs with many valid forms, where no exact reference exists; introducing it on a single-reference task adds cost and a whole class of biases for no benefit. Reserve the judge for where it's actually needed.

Related chapters