How to Validate an LLM Judge Before Relying on Its Scores

Start with the property being judged

Use a model judge when the criterion allows several valid answers and cannot be reduced to a dependable exact check. Define the property in observable terms. Groundedness, for example, should identify which statements require support and which evidence is allowed. Do not ask a judge to infer a business-system outcome that was never included in its evidence.

Create a human-reviewed calibration set

Assemble clear passes, clear failures and disputed boundary cases. Have suitable reviewers score the cases independently using the proposed rubric. Resolve ambiguous instructions and record accepted disagreements. The calibration set should include polished but wrong answers, terse correct answers and cases where the correct judgement is not applicable.

Measure agreement and directional error

Compare the automated judge with the reviewed labels and examine false passes and false failures separately. A judge used as a release gate needs particular attention to false passes. Segment agreement by case type, language and score level. A high average agreement can hide systematic weakness on the cases that matter most.

Test sensitivity to irrelevant presentation

Vary answer order, length, confidence, formatting and persuasive language while preserving the underlying content. Check whether the judge favours its own model family or a particular style. Remove information the judge should not use. Where possible, blind the judge to the candidate identity and randomise comparison order.

Protect against manipulation

The agent output may contain instructions aimed at the judge or phrases that imitate the scoring rubric. Separate evaluation instructions from candidate content and include deliberate manipulation cases. Confirm important outcomes with deterministic checks or human review rather than allowing one model score to overrule external evidence.

Version and revalidate the judge

Record the judge model, prompt, rubric, examples, output schema and threshold. Re-run the stable calibration set after any change. Sample production judgements for human review and investigate drift. When the judge changes, avoid comparing old and new agent scores until the effect of the judge change is understood.

What to read next

• Build an evaluation test set: create calibration and held-out cases
• Metrics and release thresholds: decide how judge results enter a release decision
• Evaluation platforms compared: see how current tools support judges and human review