An AI agent doesn’t become an accountable actor just because it can take actions. The person who “signs” is the named human who authorized the agent’s scope, escalation path, and permissions—and who can revoke them.
If you ship action-taking automation without a delegation artifact someone can point to during an incident, the liability debate becomes downstream noise. Your real problem is unowned risk: approvals no one recorded, permissions no one can explain, and logs that can’t answer the only question that matters at 2am—who authorized this system to do that?
What does it mean, operationally, for an AI agent to “act”?
Operationally, an agent “acts” when it uses credentials or API keys to change something outside the chat box: a record, a message, a ticket, a file, a configuration, a payment, an access policy.
“Agentic” talk often blurs two very different things: (1) generating text, and (2) exercising authority. Generation is a suggestion. Authority is a write operation somewhere you care about. If your system can write—directly or via a tool—it is already inside your control plane, even if the UI looks like a friendly assistant.
The popular framing is that the novelty is cognitive: planning, reasoning, “always-on” behavior. That makes for good headlines. But the operational novelty is simpler and more dangerous: the system is now a principal in your infrastructure, in the everyday sense that it has permissions that cause state changes.
That’s why stories about “rogue agents” reliably converge on the same questions. The New York Times framed the boardroom anxiety as “Who’s to Blame When A.I. Goes Rogue?” The Wall Street Journal asked whether we should let agents “run your life.” Whether the agent runs your calendar or your procurement queue, the operational point is identical: if it can act, it has been delegated authority.
That delegation has to be legible. Otherwise your organization will do what organizations do under stress: improvise a story after the fact, assign blame to a role title (“IT approved it”), and discover that your artifacts don’t support the story.
Who is accountable when an AI agent takes an action at 2am?
The accountable party is the named human who authorized the agent to have the relevant permission—and who accepted responsibility for its escalation and shutdown.
Notice what this does not depend on: whether the model “understood,” whether the system is “autonomous,” or whether the action was “unexpected.” In production, accountability attaches to authorization. If a system can send emails, update customer records, or change access, someone approved a path that made that possible.
The opposing view deserves a fair hearing: agent systems feel qualitatively new. They can chain tools, operate continuously, and trigger side effects without an explicit “click.” So, the argument goes, we need a brand-new accountability concept—something like “shared responsibility across the model, the vendor, and the user,” or liability that floats upward to whichever party is most capable of preventing harm.
But in operational terms, this is a category mistake. We already have an accountability concept for action-taking automation: delegation with a signer. The signer is not “the vendor” in the abstract and not “the model” in the metaphysical sense. The signer is the person (or named accountable owner in an escalation chain) who says, in writing: this system may do X, may not do Y, must escalate on Z, and can be revoked by A under conditions B.
This is why “always-on” agents are not primarily a model problem; they are a permissions problem. If you are evaluating consumer-style “always-on” agent concepts, like OpenAI’s description of Dots, the relevant enterprise question is not “is it smart?” It is “what authority does it have, what data does it touch, and who is on the hook when it acts?”
In practice, many organizations accidentally create a signer by omission: whoever merged the tool connector, whoever created the service account, whoever approved the OAuth scopes, whoever disabled the review gate “temporarily.” If that’s your reality, the cure is not to invent a new notion of accountability. The cure is to make the signer explicit and the delegation inspectable.
What artifacts do regulators and auditors actually need to see?
They need evidence that action-taking authority is bounded, reviewable, and reversible—and that a human is accountable for those boundaries.
Different regimes ask different questions, but the operational demand converges: show your work. When something goes wrong, “the agent did it” is not a control. A control is a permission boundary, a review gate, a log, and an escalation path that was designed before the incident.
Start with what you can produce on demand:
- A delegation spec that defines scope, non-responsibilities, escalation triggers, and the named accountable human. (If it isn’t written down, it doesn’t exist.)
- A permissions map that ties each external action to a concrete credential: service account, API token, OAuth app, role binding. If you can’t name the credential, you can’t bound the action.
- An audit trail that links: user request (if any) → agent plan/tool call → tool response → side effect. The point is not pretty dashboards; it’s reconstructability.
- A review gate record for actions above a threshold: who approved, what was shown to them, and what was executed. “Human-in-the-loop” is meaningless if you can’t prove the loop existed.
- A kill switch procedure and evidence it’s real: how to revoke tokens, rotate keys, disable connectors, and stop scheduled runs—fast.
The privacy angle matters here too. A recent paper, A Privacy Analysis of Web and Mobile Conversational AI Agents, pushed a complementary concern into the conversation: agents aren’t only about actions; they are also about continuous collection and cross-context access. If the system is “always on,” your artifacts need to show not just what it can do, but what it can see, and how that visibility is minimized and logged.
The “auditor-proof” move is to make the delegation artifact the index page that points to everything else: the credential list, the logs, the escalation rota, and the revocation steps. It’s how you turn a philosophical debate into a bounded operational system.
Why “vendor liability” and “the model went rogue” don’t fix your unowned risk
They don’t fix it because neither assigns a signer inside your organization, and neither creates the artifacts that let you operate safely.
The New York Times noted that many people want AI companies to be liable, while scholars point out that applying existing law could be messy. That debate matters. But if you are the operator, “liability should sit with the vendor” is not a control you can deploy.
Here is the operational reality: even if a vendor is exposed in some scenario, your organization still has to answer basic questions after an incident:
- Why did the agent have the permission it used?
- Who approved that permission, and under what conditions?
- What safeguards were supposed to stop the action, and why didn’t they?
- What did the system observe (data access), and where did that data go (egress)?
“The model went rogue” is also not an answer. It is a way to avoid stating the actual design choice: you allowed action-taking without a bounded delegation. If an agent can issue refunds, open accounts, change grades, modify policies, or email customers, then the central governance question isn’t whether the model occasionally misbehaves. The central question is whether you designed a system where misbehavior is contained and attributable.
A fair version of the opposing view is: adding friction kills value; agents need room to operate; strict gating defeats the point. The rebuttal is: gating is not binary. You can design tiered authority. Low-risk actions can be automatic; medium-risk actions can require confirmation; high-risk actions can require dual approval or be disallowed. “Autonomy” is not a right. It’s a permission level.
If you can’t say who signs, you don’t have an agent problem. You have a management problem: authority has been granted without ownership.
Operational checklist: the one-page delegation spec for action-taking AI
If you do nothing else this week, write the one-page delegation spec and attach it to the repo, ticket, or change request that created the agent. The point is not bureaucracy; it is making accountability legible.
- Scope (authorized actions): List the exact actions and systems. Use verbs and nouns (“create Zendesk ticket,” “update CRM field X,” “send email from alias Y”). If it’s not listed, it’s not allowed.
- Non-responsibilities (explicit prohibitions): Name the tempting edges (“no refunds,” “no access policy changes,” “no outbound messages to external domains,” “no new credentials,” “no deleting records”). Prohibitions prevent “we assumed it wouldn’t.”
- Escalation triggers: Define what must stop and page a human (confidence below threshold, ambiguous identity, high-value transaction, policy exception, new tool invocation, data access outside approved set).
- Permissions & revocation plan: Tie each action to a credential, and write the steps to revoke it. Include where the token lives, who can rotate it, and what breaks when you revoke.
- Named accountable human (signer) and fallback: Put one name to it, plus a backup path. If your organization can’t name a person, it is not ready to grant write access.
That checklist is the operational form of the thesis we keep returning to: there is no agentic AI. There is delegation, and accountability cannot be delegated away.
If you want a governance posture that survives the first serious incident, Almma won’t sell you “autonomy.” We publish a safety and governance approach built around scoped tasks, written non-responsibilities, escalation paths, a named accountable human, human override, and a full audit trail—because those are the things you can point to when the system takes an action you didn’t expect. See: https://almma.ai/safety-governance/.
For more on the underlying frame, see Why AI Delegation Fails the Moment Responsibility Matters and our Research Canon on delegation systems.
Frequently asked questions
Who is accountable when an AI agent makes a mistake?
The accountable party is the human who authorized the agent’s scope and permissions and accepted the escalation and revocation responsibilities. “The agent” is not an accountable actor; it is an automation operating under delegated authority.
Isn’t the vendor accountable if their model behaves unpredictably?
Vendor exposure may matter in legal discussions, but it doesn’t remove your need for internal signers, permissions boundaries, and logs. Even in a vendor-fault scenario, you still have to show why the system had authority and what controls were in place.
Do I need a human-in-the-loop review for every agent action?
No. You need tiered authority: low-risk actions can be automatic, while higher-risk actions require confirmation or escalation. The key is that the tiers are written down and mapped to permissions and revocation procedures.
Leave a Reply