LLM evaluation metrics: which ones to track and why

Pick LLM evaluation metrics by task, then add two that apply everywhere: hallucination (faithfulness to the given context) and a check that a human would accept the output. Comet's metrics guide puts it this way: "The right LLM evaluation metric depends on your task." Use heuristic scores for cheap regression checks, LLM-as-a-judge scores for meaning, and human review for the calls that matter.

The two kinds of automated metric

Automated LLM evaluation metrics are either heuristic or LLM-as-a-judge. Comet's guide to LLM evaluation metrics draws the same split: heuristic metrics are deterministic and often statistical, while LLM-as-a-judge metrics are non-deterministic and use one model to grade another.

Heuristic examples are ROUGE, BLEU, BERTScore, Levenshtein ratio and perplexity. They are fast, free to rerun and easy to explain. They break on open-ended work, because two good answers can share almost no words. Judge examples are G-Eval, answer relevance, moderation and hallucination. They read meaning, but they carry bias and prompt sensitivity.

My rule: use a heuristic metric only where there is one right answer or a reference you trust. Use a judge where wording varies. Do not report either as the truth. Report it as a signal that tells you where to read outputs.

Match the metric to the task

Start from what the output has to do, then pick the score. The table is my condensed version of the task mapping in the Comet and Nexla guides.

| Task | Start with | Add | |---|---|---| | Summarization | ROUGE, G-Eval | Faithfulness to the source | | Translation | BERTScore, Levenshtein ratio | A meaning judge such as GEMBA | | Chatbot or support | Answer relevance, moderation | Task completion | | RAG | Faithfulness, answer relevance | Context precision and recall | | Structured output | JSON validity | Schema checks on field types | | Agents | Tool correctness, task completion | Human review of failures |

Nexla's LLM evaluation guide adds the metrics that matter once an LLM feeds other software. JSON correctness is pass or fail on whether the output parses, and tool correctness checks whether an agent called the right function. Both are cheap to automate and both catch failures that a fluency score never sees.

Metrics that apply to every task

Track hallucination on everything. The Comet metrics guide calls hallucination and perplexity "table stakes" for any application. I would keep hallucination and drop perplexity unless you are comparing models, because hallucination is the one your customer will notice.

Nexla gives a worked faithfulness example. A model summarizing Einstein from supplied context made three claims, two supported and one (a Nobel Prize year) absent from the context, so faithfulness scored 2/3. The useful part is the unit: you score claims, not whole answers. A long answer with one invented claim is still a failed answer in many jobs, so I add a hard rule next to the ratio. Any unsupported claim in a regulated or customer-facing output is a fail, whatever the average says.

Model metrics versus system metrics

Score the whole system, not only the model. Comet's LLM evaluation guide separates model evaluation, which measures the LLM alone, from system evaluation, which measures the model inside your product. It argues that "a well-performing model can still result in a poor user experience if the system around it isn't properly evaluated."

Public benchmarks sit on the model side. Nexla describes MMLU as multiple-choice questions across 57 subjects, and ARC as 8,000 multiple-choice science questions from grades 3 to 9. GLUE and HELM round out its list. They help you shortlist models. A multiple-choice science quiz says little about whether your prompt, retrieval and guardrails work together.

A 2025 arXiv paper, Evaluating LLM Metrics Through Real-World Capabilities, makes the mismatch explicit: models "continue to be evaluated on benchmarks such as MMLU" that prioritise abstract problem-solving, while people use them for summarization, writing and review. Read that as a reason to build your own test set from real inputs.

What a judge cannot be trusted with

Nexla warns that using models as evaluators "carries a risk of bias, especially when the judge and the subject model share the same architecture or training data." Comet says to treat judges as a first-pass filter, not the final word.

Two things I would add. First, never let a judge grade its own model family on a launch decision. Use a different model, or a human sample. Second, calibrate before you trust: have a person label 30 to 50 real outputs, run the judge on the same set, and read every disagreement. If the judge and the person split often on the cases you care about, rewrite the rubric before you use the score.

A one-page scorecard to copy

Paste this into a doc and fill it in before you ship any LLM feature.

LLM EVAL SCORECARD
Feature:            ______
Owner:              ______
Date / version:     ______

1. Task definition (one sentence of what "good" means)
2. Test set: ___ real inputs, ___ edge cases, source: ______
3. Task metric:      ______  (heuristic or judge?)  pass line: ___
4. Faithfulness:     claims supported / claims made, pass line: ___
5. Hard-fail rules:  (any unsupported claim, invalid JSON, wrong tool)
6. Judge model:      ______ (different from the app model? Y/N)
7. Judge calibration: ___ human-labelled outputs, disagreements read? Y/N
8. Human review:     who reads failures, how often, sample size ___
9. Rerun trigger:    prompt change, model change, retrieval change
10. Ship / hold decision and reason:

Rows 5, 7 and 9 are the ones most teams skip. Hard-fail rules stop an average from hiding a bad failure. Calibration keeps the judge honest. The rerun trigger makes evaluation continuous, which Comet's guide argues is needed because behavior drifts with prompt changes, user behavior and system updates.

Keep a person in the loop

Every workflow I build starts with a real input, produces an output someone can inspect, and keeps a human review step before it ships. Metrics tell you where to look. They do not make the call. For agents, I wrote up the same approach in AI agent evaluation, and the kits in the quality and production collection are plain files built around that review step.

Next step: copy the scorecard, fill rows 1 to 3 for the one LLM feature you ship most, and label 30 real outputs by hand before you pick a single metric.

How this post was made: drafted with AI assistance (Claude Sonnet 5.5) from the sources linked above, then checked by Claude Opus 5.5 and an automated quality gate that matched every figure and quotation to its source on 2026-10-28.