Audit Trail vs. Decision Record: A Four-Question Test for Your AI Agent

Two people will eventually ask about the same agent decision, usually weeks apart.

The first is an auditor, or a security reviewer, or whoever at a customer is filling in your vendor questionnaire. They want to know whether your controls ran: was the action checked against a rule, by something other than the agent, and is there a record of it? Not long ago, plenty of teams running agents couldn't answer that. More of them can now, and that's real progress.

The second person is the one the decision landed on, or the operator who owns the workflow and just heard from them. They want to know why the agent did that. The audit trail usually can't tell them, and most teams find that out at the moment someone asks.

Below is a test you can run on one decision from last week to see which of those two people your records can answer. It needs one decision and the systems you already have.

A composite, to show the shape rather than report a case.

A company sells kitchen appliances online and uses an agent to handle warranty claims. The policy is simple. A claim within twelve months of purchase gets a replacement; anything older is declined with an offer of paid repair. Before a decision goes out, a policy check confirms it fits the rule.

A customer files a claim for a stand mixer and attaches two documents: her original order confirmation, fourteen months old, and a receipt from eight months ago for the replacement unit the company shipped her after an earlier fault. The agent takes the purchase date from the order confirmation and declines the claim. The policy check confirms that a fourteen-month-old purchase is outside warranty. The decline email goes out exactly as approved.

Two days later she replies, unhappy, and a support lead reverses the decision and ships a new mixer.

Hold on to this one. The test below runs on it.

Start with the strongest version of the current approach, because it's good.

A runtime check, evaluated at the moment of action against a written rule by something the agent doesn't control, produces one of the best records a team can have: which rule applied, whether the action was allowed or blocked, and any evaluation output attached to it. It was written at the time, not reconstructed later. When an auditor asks how you know the agent can't approve refunds above its limit, that record is the answer, and it's usually the first thing they ask.

Trace review is the other route, and it gets better with use. Someone reads a sample of runs and corrects the ones that look wrong, and the corrections are kept as precedent, so the next reviewer (or the next revision of the prompt) inherits what the last one learned. Over a few months that adds up to real knowledge about where the agent tends to slip.

If you have both, you're ahead of most teams running agents. Keep them.

Go back to the mixer.

The check's record says the decline was allowed under the twelve-month rule. That's accurate. But it's a record of the check's decision, that this decline fit the rule. The agent's decision was something else: that the original order set the purchase date and the replacement receipt didn't count. That choice is the whole case, and the check never weighed it, because by the time the check ran it was handed a date the agent had already picked.

Trace review stops in a different place. It covers the runs someone read. If the mixer claim wasn't in the sample, nobody reviewed it. If it was, the reviewer saw a fourteen-month-old order and a decline that matched policy, and moved on. Each review also starts cold. Unless someone connected them, the reviewer has no way to know this customer already had a replacement unit, or that the same misreading happened on other claims that month.

The reversal has a problem of its own. It records that a person intervened, which is not the same as recording that the agent was wrong. The support lead may have reversed it because the agent misread the receipts, or because the customer was upset and a mixer is cheaper than the conversation. A correction is evidence that someone with more context chose differently. Treated as ground truth, a pile of them can teach a team that the agent's mistake is saying no to anyone who complains.

That's the distinction the title is drawing. A plain audit trail is about the controls. A Decision Record is an account of a single agent decision: what the agent took the situation to be, what it had to choose from, what it chose, which checks it passed, and what happened afterward, kept together so you can ask questions about the choice and not only about the controls around it.

Pick one consequential decision your agent made last week. It doesn't have to be a failure, just a decision that mattered: money moved, a customer was told no, something was escalated or wasn't. Answer four questions about it.

For each answer, also write down where it came from: a record written at the time, something you rebuilt today from logs or transcripts, or someone's memory.

That second column is the point of the exercise. You're not grading the agent. You're finding out which questions about its decisions your systems can answer without a person reconstructing the run, and the answer is almost never the same for all four.

Here's how the mixer comes out for a team with good controls:

Questions one and three come back clean because the systems were already built to record them. Questions two and four are where it thins out: one is reconstruction, the other is memory. It's a common shape, and it's the gap between an audit trail and a Decision Record, visible in four rows of your own data.

Once you have the table, resist averaging it. Two clean answers don't make up for a decision nobody recorded, because they answer different questions. Resist reading the reversal as the verdict, too. The outcome tells you the decision didn't hold. Whether the agent should have used the replacement receipt is a question about how the workflow is supposed to work, and the table just showed you that nobody has answered it yet.

One decision is an anecdote. It tells you which questions your records can answer, not how often the agent gets things wrong. Run it on a few decisions of different kinds before you draw conclusions about the agent itself.

"Unknown" is a fair answer. Plenty of decisions never get a reported outcome. Writing "unknown, nothing recorded" beats leaving the cell blank, because it tells the next person exactly which question is still open.

It doesn't replace your controls. If question one comes back with no check at all, fix that first. The test is for the questions that are left once the controls exist.

Start keeping one link. When someone reverses, edits or escalates an agent decision, put the run's ID in the ticket or note where they did it. It's a small habit, and it moves question four from memory to a record, which makes it the cheapest of the four gaps to close. Run the test again on a decision from next week and see which column changed.

Question two, the agent's own decision, is the one I'm building Nalyqor to keep on the record.