Oversight ROI Depends on a Rate Nobody Measures

Human review of AI output is treated as a control rather than as an investment, which is why almost nobody asks what it returns. The cost is real and visible in salary, in throughput and in latency. The benefit is asserted.

‍

The break-even test in circulation says review pays where oversight cost is below the error rate multiplied by the cost of an uncaught error. The arithmetic is right and one term is missing, and its absence flatters oversight considerably.

‍

Review Only Prevents What the Reviewer Notices

‍

An error the reviewer approves costs exactly what it would have cost unreviewed, plus the price of the review. So the benefit is not the error rate multiplied by the loss, it is the error rate multiplied by the detection rate multiplied by the loss.

‍

Detection rate is the variable nobody has. Programs measure how many items were reviewed and how quickly, which describes throughput rather than efficacy. A review step catching half of what passes through it returns half of what the standard formula predicts, and a step catching very little returns the cost with no offset.

‍

The Signature Is Already Visible in Most Programs

‍

A near-zero override rate combined with a very short median review time is the observable indication that detection is low. Both numbers are usually available and rarely read together, and establishing whether review is genuine is the prerequisite for pricing it at all. There is no point computing a return on a control that is not operating.

‍

The Reviewer Can Also Introduce Error

‍

The second missing term is negative. A reviewer who rejects correct output imposes a cost, whether through rework, delay, a lost transaction or a customer outcome that would have been fine.

‍

Oversight therefore adds value only where the reviewer is more accurate than the model on the population being reviewed, which is an empirical question with an uncomfortable answer in some deployments. Measuring override accuracy against later ground truth is the only way to establish it, and the exercise occasionally shows a review step making outcomes worse while satisfying a control requirement.

‍

The Honest Expression

‍

Written out, the value of a review step is the loss prevented, being error rate times detection rate times cost of an uncaught error, minus the loss introduced, being false rejection rate times the cost of wrongly rejecting good output, minus the direct cost of reviewing, minus throughput and latency costs. Four terms, two of which the standard framing omits, and both omissions run in favor of reviewing.

‍

Latency Costs More in a Chain Than in a Queue

‍

Direct cost and throughput are straightforward to estimate. Latency is where agentic deployments differ from classification workflows and where the figure is usually understated.

‍

A human gate inside an agent chain does not cost the review time. It costs the idle time of every downstream step waiting on the decision, and where the chain holds resources or locks, it costs those too. A four-minute review inside a workflow that would otherwise complete in nine seconds has changed the economics of the whole workflow, which is a different calculation from adding four minutes to a batch process.

‍

Asynchronous Review Changes the Term

‍

Where the action can proceed and be reversed, review can happen after execution and the latency cost largely disappears. Where it cannot, the gate is synchronous and expensive. Classifying each action by reversibility therefore determines which review costs what, and the same reversibility map does work in an incident.

‍

Uniform Review Is Negative on the Bulk and Positive on the Tail

‍

The cost of reviewing an item is roughly constant. The loss prevented varies by orders of magnitude across the same population, so a single review policy applied to everything is simultaneously wasteful and insufficient.

‍

The routing threshold falls out of the arithmetic. Review where detection rate multiplied by the cost of an uncaught error exceeds the per-item cost of reviewing. Below that line, review is a net loss and the money is better spent on constraining what a wrong output can do, which scope rather than filtering argues for elsewhere. Above it, review is underprovisioned in most organizations because attention is spread evenly.

‍

Concentrating Attention Raises the Detection Rate

‍

A useful second-order effect. Routing fewer items to a reviewer with more time per item improves detection on those items, so selective review does not merely save the cost of the items it skips. It increases the return on the items it keeps, so the two adjustments compound rather than trading off.

‍

Where the Arithmetic Does Not Decide

‍

Some oversight is required regardless of what a model says it returns, and pretending otherwise produces an analysis nobody can act on.

‍

Regulatory obligations mandating human oversight for certain decisions are not trade-offs. European rules on high-risk systems require oversight capable of catching anomalies and reversing outputs, and drug regulators expect an adequacy determination by people independent of the development team. In those cases the arithmetic still helps, because it stops answering whether to review and starts answering how to make a mandatory review effective at the lowest cost, which is a better use of it anyway.

‍

Mandatory Does Not Mean Unmeasured

‍

A required review step with a detection rate nobody has measured satisfies the obligation on paper and produces the exposure the obligation was written to prevent. Measuring it is what allows a program to defend the arrangement rather than merely evidence its existence, and an examination testing whether a control operated are asking about substance rather than about the step.

‍

Five Numbers Populate the Model

‍

All five are obtainable and four of them most programs already generate without reading.

‍

• Per-Item Review Cost: Median review time multiplied by loaded reviewer cost, plus the tooling overhead per item.
• Override Rate: How often the reviewer disagreed, which bounds detection from above.
• Override Accuracy: Whether the reviewer was right when they disagreed, measured against later ground truth.

‍

The remaining two are cost of an uncaught error, which comes from the loss model rather than from the review process, and detection rate, which is the difficult one. Detection cannot be measured from the reviewed population alone, because that population is where detection already happened. It requires either a sample deliberately left unreviewed and checked later, or known-incorrect items seeded into the queue, which testing on a cadence rather than once makes routine.

‍

Seeding Is the Practical Route

‍

Leaving a live sample unreviewed is unacceptable in most regulated contexts, which leaves seeding. Inserting items with known defects and measuring how many are caught produces a detection rate directly, and it is the same exercise that tests whether review is genuine, so it answers two questions for one effort.

‍

Price It Like Any Other Control

‍

Human review is a control with an unusually visible cost and an unusually unexamined benefit. The circulating break-even test omits the detection rate, which reduces the benefit by however much reviewers miss, and omits the cost of wrongly rejecting correct output, which can turn the return negative. Adding both terms usually argues for reviewing fewer items more carefully rather than for reviewing less, and it identifies the population where review is currently a net loss. Kovrr's AI risk quantification supplies the loss figure the calculation needs, which is the term no operational measurement produces.

‍

To see modeled loss per AI system alongside the controls that reduce it, book a demo mapped to your own estate.