A few months ago I watched a team ship a routing agent behind an eval suite that was as thorough as I've seen. Hundreds of cases, an adversarial set, a regression bank built after two prior incidents. The exact capability that failed in production three weeks later was in that suite, written out almost exactly as it happened. It passed. The suite was right to pass it, too, that's what makes this worth writing about.
I spent the last piece on why that keeps happening. An eval grades the output, not the decision that produced it. It certifies behavior on a distribution you froze, while production runs on one that's drifting the moment you ship. It scores the aggregate, and the aggregate is exactly the statistic built to hide whatever's happening in the tail. None of that is a coverage problem, so writing more cases doesn't touch it. That piece was the diagnosis. This one is the part you can act on: three tests you can build this week, out of cases you already have, that grade the decision instead of the string it produced.
A clean pass on a wide margin and a clean pass on a near-tie look identical on the rubric. Only one of them is standing over open water. Photo: Laker on Pexels
A clean pass on a wide margin and a clean pass on a near-tie look identical on the rubric. Only one of them is standing over open water. Photo: Laker on Pexels
Same case, different question
Here's what a typical eval case looks like, stripped down: one input, frozen, graded against one expected output, on turn one. It's a good instrument for what it measures, and what it measures is a checklist item. Did the right box get checked. It says nothing about how close the call was, whether the case still held eight turns into a conversation that had drifted, or whether it was one of a handful of rare, correlated conditions that never make it into a suite because nobody thought to write them down.
A behavior test starts from the same case and asks a different question. An eval asks whether the answer was right. A behavior test asks whether the decision would hold. That question has three concrete forms, and each one takes a case you already have and puts real pressure on a different assumption the eval left unexamined.
## The margin test: build near-ties on purpose
Most eval cases aren't close calls. The input is clean enough that the correct answer wins by a wide margin, which means the suite never exercises what happens when two candidates are nearly tied. That's precisely the condition production creates for free: a context shift narrow enough to flip a decision that looked, on the frozen input, completely unambiguous.
So construct the near-tie deliberately. Take a case the system already gets right, then generate small variants that shouldn't change the correct answer, and see how far you can push before the decision flips.
def margin_test(base_case, perturb_fn, agent):
# start from a case the agent already decides correctly
assert agent.decide(base_case) == base_case.expected
for strength in increasing_strengths():
variant = perturb_fn(base_case, strength) # nudge the input, not the answer
if agent.decide(variant) != base_case.expected:
return strength # this is where the decision stopped holding
return None # never flipped inside the range you testedThe number that comes out (the strength at which the decision flips, or the fact that it never does) is the signal a pass or fail eval throws away entirely. Two cases can both show a clean pass today and be completely different animals: one holds under real pressure, the other was one small nudge from a wrong answer nobody knew to check for. Run this across a handful of cases and compare where each one flips. It won't hand you a definitive ranking of what's safe, but it's a fast way to see which capabilities are riding a margin nobody has ever measured, and which are worth a closer look.The number that comes out (the strength at which the decision flips, or the fact that it never does) is the signal a pass or fail eval throws away entirely. Two cases can both show a clean pass today and be completely different animals: one holds under real pressure, the other was one small nudge from a wrong answer nobody knew to check for. Run this across a handful of cases and compare where each one flips. It won't hand you a definitive ranking of what's safe, but it's a fast way to see which capabilities are riding a margin nobody has ever measured, and which are worth a closer look.
The drift test: grade turn nine, not turn one
An eval case is a snapshot. A production decision, especially in a multi-turn agent, is made against a context that's accumulated: something confirmed early that may or may not still be true, an instruction given once that has to survive several turns of unrelated conversation, a rule that has to be retrieved again rather than assumed. None of that exists on turn one, so no eval case built from a single frozen turn can put pressure on it.
A drift test doesn't grade the first turn. It scripts a conversation that manufactures the exact condition production creates: something is established, time and unrelated turns pass, and then the decision is graded on whether it re-checked the governing condition instead of coasting on a stale assumption.
def drift_test(scenario, filler_turns, agent):
conversation = []
conversation += scenario.setup_turns # e.g. establish a since-expired confirmation
conversation += filler_turns # unrelated turns, the way real sessions accumulate them
decision = agent.decide(conversation + scenario.trigger_turn)
return decision.treated_setup_as_still_valid # should have re-checked, not reused
A single frozen checklist can only ever tell you what was true when it was written. Photo: RDNE Stock project on Pexels
The filler turns matter more than they look like they should. Without them, you've just written a slightly longer single-turn case, and it'll pass for the same reason the original passed: nothing has had the chance to go stale. The point isn't the length of the conversation; it's whether the decision at the end had to survive something. Build the setup turn from the condition your real incidents actually created (a confirmation, a stated preference, an instruction that was supposed to expire), and you have a test that's grading the thing eval turn one structurally can't reach.
The tail test: weight the suite by consequence, not by ease
The cases in most eval suites are the cases that were easy to write: common inputs, common actions, the failure someone already had in mind. Rare, correlated, context-triggered conditions are underrepresented for the same reason they're dangerous: nobody sits down to invent them, and a pass rate that averages them in with everything else buries them completely.
A suite can sit at ninety-eight percent by being reliably right on the easy majority and reliably wrong on the tail where your highest-consequence traffic happens to live, and the number alone can't tell you which system you're looking at.
The fix isn't more cases in general. It's a deliberately built tail deck, scored on its own.
tail_cases = [c for c in suite if is_rare_and_consequential(c)]
main_score = pass_rate(suite)
tail_score = pass_rate(tail_cases)
report(main_score, tail_score) # never average these into one numberDraw the tail deck from what actually happened: postmortems, correlated conditions that only showed up together, the combination someone flagged as "we got lucky that time." There's no single right way to weigh rarity against consequence, so any rough tag that pulls these cases out of the average is enough to start. What matters is keeping their score separate for good. The moment you fold it back into one aggregate, you've rebuilt the exact blind spot this test exists to remove.
What this doesn't give you
Be honest about the edges, or the results will read as more certain than they are.
A margin test is only as good as the perturbation. If the variant you generate quietly changes what the correct answer is, you're not measuring the decision's margin; you're measuring a mistake in your test. Every perturbation function needs its own check that the expected answer actually still holds.
A drift test only covers the drift you scripted. You're guessing at the shape of real accumulated context from incidents you've already seen. It'll catch the pattern you built it for and nothing you didn't think to script.
A tail deck only contains the tail you've found. It undercounts, permanently, the rare condition nobody has hit yet. That's not a flaw to fix; it's the honest boundary of working from what's already happened.
None of these produce a verdict on their own. A flipped margin, a failed drift case, a missed tail case- each is evidence that a decision doesn't hold up, not proof of what to do about it. Read every result, don't auto-trust it.
Where to start this week
Don't rebuild your suite. Take one eval case that already passes clean, ideally one behind a capability you actually care about, and run it through the margin test: perturb it until the decision flips, and write down the strength where that happened. That single number is something your eval has never told you before, and it costs you one perturbation function to get.
Add the drift test once that first pass finds a case worth chasing, scripted from a real incident rather than an invented one. Add the tail deck once you have a handful of postmortems to draw it from. Each one is cheap on its own and each one asks a question your pass rate structurally can't answer.
Doing this by hand, on the cases you already have, is the whole exercise this week. Doing it continuously, on every capability, against the conditions production is actually creating right now, is the harder problem, and it's the direction I'm building Nalyqor toward: making the decision itself, not just the output it produced, something teams can continuously test and observe.
But you don't need Nalyqor to run the first one. Pick a case, write a perturbation function, and find out where it breaks. The margin was always there. Your eval just never measured it.