How to Evaluate an LLM App Before Users Do

Replace “the answer feels good” with task-specific criteria. For extraction, score field accuracy. For support, score resolution and policy compliance.

Keep a small human-reviewed gold set and a larger rolling set drawn from real usage.

Include missing context, conflicting instructions, prompt injection, sensitive data, long inputs, unusual language, and requests the product should decline.

Evaluate the entire workflow—including retrieval and tools—not just the final sentence.

Quality, latency, cost, and risk trade against one another. Record all four for every release.

Set release thresholds before running the evaluation so results cannot move the goalposts.

Group failures by root cause. Fix deterministic issues with code, missing knowledge with data, ambiguous tasks with interface design, and probabilistic behavior with model changes.

Preserve failed examples as regression tests.