Who's Accountable When an AI Agent Makes the Wrong Call?

On a Tuesday morning in Q3, a procurement agent at a mid-market manufacturer approved a $340,000 payment to a vendor account. The vendor name matched the approved-vendor list. The invoice format matched the standard template. The agent verified both, cross-checked the amount against historical purchase orders, and released the payment through the treasury API within eleven minutes of the invoice arriving. No human touched the transaction. Three weeks later, the real vendor called to ask about the outstanding invoice. That was when the finance team learned they had paid a fraudulent lookalike account with a routing number that had been created seventy-two hours before the invoice landed.

‍

The internal review moved fast, then stalled. The procurement team pointed at the agent's developer, arguing the verification logic should have caught the routing mismatch. The developer pointed back at procurement, arguing the agent had performed exactly as specified and the approval threshold above $250K was set by the business owner. The finance team pointed at the vendor onboarding process. The CISO pointed at everyone. Six weeks later, the CFO was still asking a question the organization could not answer cleanly. Who, specifically, was accountable for the $340,000 loss?

‍

The Question the Scenario Raises

‍

The scenario is fabricated. The pattern is not. Every enterprise operating agentic AI at scale is one badly designed autonomous decision away from a version of this conversation, and most of them will fail to answer the accountability question with any confidence. The answer matters for three reasons. Legal exposure sits with someone regardless of who the enterprise names. Regulatory disclosure obligations under frameworks like the EU AI Act require documented accountability structures before the incident, not after. And the internal governance of the AI program either survives its first serious autonomous failure or does not, depending on whether the accountability chain was designed in advance or improvised in the aftermath.

‍

The AI accountability discipline exists specifically because software cannot be held accountable. An autonomous AI system executing a task has no legal standing, no financial liability, no professional license to lose, and no meaningful way to answer to consequences. Accountability has to sit with humans and the organizations they represent, and the specific humans have to be named before the agent makes its first autonomous decision.

‍

The Three-Tier Accountability Chain

‍

When an autonomous AI system produces a bad outcome, liability typically fractures across three parties. Understanding all three is prerequisite to structuring the governance program that prevents the fracture from becoming a crisis.

‍

Tier One: The Deploying Organization

‍

The company that authorized the agent to act carries primary liability in most corporate scenarios. Legal doctrines like vicarious liability treat the AI as a tool operating on behalf of the organization, which means the organization inherits full responsibility for the agent's decisions. Courts have consistently ruled that companies must honor the actions their AI systems take, including promises made to customers, contracts signed, and financial commitments executed. In the procurement scenario above, the manufacturer holds primary liability for the $340,000 payment regardless of what the agent's developer intended or what the vendor onboarding process should have caught.

‍

Tier Two: The Developer or Vendor

‍

Responsibility can shift toward the developer or vendor when the failure traces to a design defect, an inadequate safeguard, or a documented capability gap that the developer represented as solved. This is where product liability doctrines become relevant, particularly under the EU Product Liability Directive and equivalent US state frameworks. If the developer represented that the agent verified vendor account routing numbers and it demonstrably did not, some liability moves back up the chain toward the developer. If the developer disclosed the limitation and the deploying organization accepted the risk, most of the liability stays with the deployer.

‍

Tier Three: The Human Operator or Approver

‍

Accountability sits with the human operator when the agent was deployed as a decision-support tool rather than a fully autonomous system, and the operator failed to verify the agent's output before acting on it. In the procurement scenario, the accountability question tightens if the workflow was supposed to include a human approval step above $250K and the finance team turned that gate off to speed processing. The human who authorized the workflow change becomes accountable for the loss, alongside the organization itself.

‍

Why "The AI Did It" Is Not a Defense

‍

New regulations explicitly block the argument that autonomous software caused harm on its own. California's AB 316 bars defendants from claiming their AI system autonomously caused the harm as a way to escape liability. The EU AI Act pushes further, requiring documented human oversight, risk management processes, and audit trails for high-risk AI systems. The regulatory direction is consistent across every serious jurisdiction. If an enterprise chooses to deploy AI to make decisions, the enterprise owns those decisions.

‍

This regulatory posture shifts the practical question every enterprise deploying agentic AI has to answer. It is no longer whether the organization can escape liability by pointing at the model. It is whether the organization can document the governance structure, oversight mechanisms, and audit trail that show the decision was designed responsibly and monitored actively. Programs that build this documentation into the deployment lifecycle produce defensible answers when the CFO asks who is accountable. Programs that skip the documentation produce the six-week internal review that never quite resolves.

‍

How Governance Programs Operationalize Accountability

‍

Effective accountability is not achieved by naming a single owner after the incident. It is designed into the AI governance program from the first day of every agent deployment. Two categories of governance mechanisms cover the fundamentals.

‍

Ownership Assignment Requirements

‍

• Named human owner per deployment: Every AI agent operating in the environment has a specific named person who is accountable for the outcomes it produces, documented in the AI risk register before the agent begins autonomous action.
• Documented approval thresholds: Explicit dollar values, transaction types, and decision categories where the agent may act autonomously versus where a human-in-the-loop approval is required before execution.
• Cross-functional accountability review: Approval workflows that involve the business owner, technical owner, and risk owner rather than any single team, so accountability is distributed rather than concentrated on one person who can be scapegoated after an incident.

‍

Runtime Governance Requirements

‍

• Immutable audit trails per decision: Every autonomous action preserved with the input context, reasoning path, tool calls, and final output so post-incident forensics can reconstruct exactly what happened.
• Real-time anomaly detection: Behavioral guardrails that flag decisions falling outside the agent's approved operating envelope, ideally through the same connected telemetry approach that discovers every agent operating in the environment.
• Escalation paths for edge cases: Explicit rules that pause execution and require human review when the agent encounters decision inputs it was not designed for, rather than allowing the agent to act at the edges of its trained behavior.

‍

What the AI Risk Register Documents for Every Deployment

‍

The AI Risk Register is where accountability lives operationally. Every agent deployment produces an entry that names the owner, describes the scenarios that could produce harm, quantifies the likelihood and impact of each scenario, and documents the response plan if the scenario materializes. Programs that maintain the register rigorously produce a clean audit trail from deployment through incident response. Programs that treat the register as optional produce the six-week internal review where no one can find the documented owner.

‍

The register also serves as the operational bridge between the technical AI program and the executive risk conversation. When a CFO asks about accountability for a specific incident, the register answers immediately with the named owner, the documented decision authority, and the response plan that governs the incident. When a regulator asks under EU AI Act obligations, the register produces the required documentation without a scramble. When the board asks whether accountability sits with the right roles across the AI portfolio, the register answers portfolio-wide rather than one entry at a time.

‍

The Post-Incident Audit Trail Every Agent Deployment Needs

‍

When an agent produces a bad outcome, the post-incident forensics depend on data that has to be preserved from the moment the agent begins operating. Two categories of audit trail cover the fundamentals.

‍

Decision Context Preservation

‍

• Input data captured at execution: The specific prompts, retrieved documents, and tool responses the agent had access to at the moment of the decision, preserved in a form the incident response team can reconstruct without depending on the agent's own memory.
• Reasoning path recording: The chain of tool calls, intermediate outputs, and reasoning steps the agent took to arrive at the final decision, since standard application logs miss most of this and generic SIEM tooling cannot correlate it.
• Identity delegation trail: The named human principal whose authority the agent was operating under at the moment of decision, preserved across every hop in multi-agent chains so accountability does not vanish when authority is delegated through the Model Context Protocol.

‍

Governance Trail Preservation

‍

• Approval workflow evidence: Documented proof of which approval workflows fired and which did not, including any workflow that was bypassed or turned off before the incident occurred.
• Guardrail enforcement history: The complete record of every guardrail evaluation the agent's action passed through, including any that returned warnings the operator did not investigate.
• Ownership continuity documentation: The specific owner of record at the moment of the incident, especially in programs where AI agent ownership changes hands across quarters or team reorganizations.

‍

Building Accountability In Before You Need It

‍

Accountability for AI agents cannot be manufactured after an incident. It has to be designed into the governance program from the first day of every deployment, documented in the AI Risk Register, enforced through named owners and approval workflows, and preserved through immutable audit trails that let post-incident forensics reconstruct what actually happened. Programs that build this structure in advance survive their first serious autonomous failure with reputation, regulatory posture, and organizational cohesion intact. Programs that improvise accountability after the incident lose all three and typically produce a leadership crisis on top of the operational one.

‍

The organizations moving fastest on this in 2026 are the ones treating AI governance as a holistic risk management function rather than a technical compliance workstream. They operate the AI Risk Register as a live accountability document rather than an audit artifact. They preserve decision context through connected telemetry that spans browser sessions, endpoints, network flows, identity providers, and AI catalog data through the AI Security and Governance Platform. And they document the approval workflows, guardrail enforcement, and ownership continuity that make the accountability chain defensible when a real incident tests it.

‍

To see how Kovrr operationalizes accountability across every AI agent operating in the enterprise, book a demo tuned to your specific agent deployments.