The Senior-Engineer Tax: What an AI Incident Actually Costs to Diagnose

A few weeks back I put six rows around what a bad AI decision costs: the call itself, the recovery loops, the downstream work that ran correctly on a wrong premise, the detection, the correction, and the scope a team quietly stops delegating afterward.

Row 4 of that worksheet was one line: engineering hours spent working backward from a symptom to the decision that caused it.

That row deserves its own page, because it behaves differently from the other five. The others are damage. This one is labor, it lands on a specific and very short list of people, and it recurs on a schedule nobody sets.

So: same exercise, different number. Not what the decision cost. What it cost to find out why it happened.

Pull up your most recent AI incident. A real one, timestamped, where something behaved oddly and somebody had to go figure out why. Four questions, in ascending order of how hard they are to answer and how much they actually tell you.

What to write down: elapsed time from "something looks wrong" to "we know why." Not to "it's fixed." To the moment somebody could explain the behavior.

Where to find it: your incident record already has both timestamps, though probably not labeled that way. The first is whenever the symptom got reported. The second is buried in a thread, usually a message that starts with "ok so."

What it tells you: less than you'd hope. This is the number everyone reaches for and it's the weakest of the four, because it blends two very different things: how long the work took, and how long the work waited. Which brings us to the useful part.

What to write down: of that elapsed time, how much was somebody actively investigating, and how much was the question sitting in a queue waiting for a particular person to be free.

Where to find it: the same thread. Look for the gap between when the question was asked and when the person who could answer it started answering. In plenty of incidents, the waiting turns out to be larger than the investigation itself.

What it tells you: whether you have a speed problem or an availability problem. They look identical on a status page and they have nothing to do with each other. If most of your clock is waiting, buying faster tooling does nothing, because the bottleneck was never the investigation.

What to write down: what didn't happen that week because the investigation did. The feature that slipped, the review that queued up, the thing that got pushed to next sprint.

Where to find it: your sprint board, comparing what was committed against what shipped in the week containing the incident. Nobody codes this to the incident, so you have to line the two up by hand.

What it tells you: the opportunity cost. This is the shape I wrote about on the blog this week, the work arriving on somebody else's timeline and refusing to be scoped in advance. Diagnosis time isn't drawn from a pool of spare hours, it comes out of committed work, which is why a two-day investigation reliably costs more than two days. This is the number that rarely appears on any incident record, and it's the one an engineering leader feels at the end of the quarter without being able to name.

What to write down: how many people in your organization could have produced that explanation. Not who was available. Who was capable.

Where to find it: one message to your engineering lead. "If both of them had been out, who else could have worked out why the agent did that?" The answer arrives in about a minute and it's usually a number you can count on one hand.

What it tells you: everything the first three numbers are actually a function of. A four-hour investigation from a bench of twenty is an operating cost. The same four hours from a bench of two is an exposure, because the same two names will appear on the next one, and the one after that.

This is the tax in the title. You are not paying an engineering rate. You are paying the rate of the specific people who can read that specific system, and that supply doesn't grow automatically when you add headcount. I went into why the bench stays small separately: most of what makes someone able to answer is accumulated familiarity with one system's history, which isn't transferable in an onboarding doc.

Has this shape been diagnosed before?

Go look. Search your incident tracker for the same symptom six or twelve months back. If you find one, ask whether the second investigation started from the first one's conclusion, or from the runs.

Too often, from the runs. The earlier investigation produced an answer, somebody adjusted a prompt or tightened a bound, and the reasoning that got them there closed with the ticket.

That's the compounding version of this cost, and it's the reason the four numbers above understate it. You are not paying them once per incident. You are paying them once per recurrence, at full price each time, because nothing carried forward.

It doesn't tell you the incident was preventable. Some weren't, and pricing diagnosis isn't an argument that the diagnosis shouldn't have happened.

It doesn't price the incident itself. That's the other worksheet, and the two numbers are unrelated: a cheap incident can be expensive to explain, and a costly one can be obvious in ten minutes.

And it is not a case for hiring. If number 4 came back small, the instinct is to make it bigger by recruiting, and that mostly doesn't work on the timeline you want. The bench is small because of what its members know about your system, not because of what they know about AI.

What the audit is for is simpler: knowing what a class of incident actually costs you, so that when you decide how much AI to run next quarter, you're deciding with the number in front of you rather than discovering it later in missed commitments.

Run it on one incident, and if you only do one of the four, do number 4.

It takes a single message and no data collection, and it reframes the other three. Bench size changes how you should read every other number on this page.

Doing this by hand, on one incident, is an audit. Having the answer already sitting there when the question comes up, because the reasoning from the last investigation was kept as something more durable than a closed thread, is the harder version, and it's the direction I'm building Nalyqor toward. The audit works without it. But it's worth noticing that the most expensive part of what you just measured is the part that gets thrown away.