Evals: turning AI quality into a number you can track
Vibes don't ship. If you can't score your AI system, you can't tell whether a prompt tweak helped or quietly broke three other things. Here's how we build evals that hold up.
Every AI team hits the same wall. The demo works, someone tweaks a prompt to fix one bad case, and a week later a different case that used to pass is failing. Nobody notices until a customer does. The system got worse and there was no number to catch it.
Evals are how you turn 'does this feel better?' into a measurement you can commit to. They are the unit tests of probabilistic software, and they are the single highest-leverage investment most teams are not making.
Start with a golden set, not a benchmark
Public benchmarks tell you how a model does on someone else's problem. What you need is a set of real inputs from your own domain, each paired with a known-good outcome. We build these from production traffic, support tickets and the edge cases that have already burned us.
A hundred well-chosen examples beat ten thousand generic ones. The set should span the boring common path, the known failure modes, and the adversarial cases you are most afraid of. Curate it by hand at first — the act of labeling forces you to define what 'correct' even means, which is usually the hard part.
Pick a grader that matches the task
Not everything can be scored with an exact-match check, and not everything needs an LLM judge. Match the grading method to what you are actually measuring, and prefer the cheapest one that works.
The mistake is reaching for an LLM judge by default. It is slower, costs money per run, and introduces its own variance. Use it only where the answer is genuinely open-ended, and validate the judge against human labels before you trust it.
- Deterministic checks for structured output — valid JSON, a value in range, a required field present
- String or embedding similarity when there is a reference answer to compare against
- Code execution and test suites when the output is code
- An LLM-as-judge, with a rubric, only for open-ended text where nothing simpler applies
- Human review on a small sample to keep every automated grader honest
Score the pipeline, not just the model
In a real system the model is one link in a chain — retrieval, tool calls, formatting, guardrails. A failing answer is often a retrieval miss, not a reasoning miss. If your eval only looks at the final text, you will spend days tuning the wrong component.
We score the stages independently. Did retrieval surface the right document? Did the agent pick the right tool? Was the final answer grounded in what it was given? When each stage has its own number, a regression points straight at the component that caused it.
Wire evals into the deploy, not a spreadsheet
An eval you run by hand once a month is a report. An eval that runs on every pull request is a safety net. We put the golden set in CI so a prompt change, a model upgrade or a retrieval tweak has to clear the same bar as any other code change.
This is what makes model upgrades boring. When a new model version ships, you do not guess whether it is better for your use case — you run the suite and read the delta. Regressions block the merge, the same way a failing test does.
Treat the eval set as a living asset
Every production incident is a missing test case. When the system gets something wrong in the wild, the fix is not just patching the prompt — it is adding that case to the golden set so the failure can never silently return.
Over time this set becomes one of the most valuable things you own. It encodes exactly what 'good' means for your domain, it makes every future model migration a measured decision, and it is the difference between an AI product you can improve with confidence and one you are afraid to touch.