Most of the time the gap gets papered over. The system executes and people decide, and when decision rights are unclear, those people fill the hole informally with an email to a manager, an exception handled by whoever happens to know the supplier, or a workaround that nobody ever writes down. It is messy and slow, and it works well enough that nobody fixes it.
An AI agent removes that buffer.
When the system itself makes the decision, there is nobody left in the process to absorb the ambiguity. You would expect the debate on agent governance to start there. Instead, most of it treats the problem as new, and reaches for new technical controls: permissions, guardrails, monitoring, approval gates, audit logs. Those controls are necessary. On their own, they govern agents as tools, and most AI strategies are betting on agents that act.
More agency means more potential value, and more need for governance. Adding human supervision doesn’t get you out of that trade-off, because if every consequential action needs a human to approve it, you end up with a safe, compliant system that is really just workflow automation with a language model attached. So the question worth asking is how much agency you want an agent to have, and which mechanisms make that much agency viable.
The old problem, with an agent in it
Take an agent that handles supplier invoices. It reads the invoice, matches it against the purchase order and goods receipt, codes it and approves it for payment.
Govern it as a tool and you get the familiar setup: read access to the ERP, write access to a single approval field, every action logged, and a human signing off on anything above DKK 50,000. The agent matches. Accounts payable still decides. The time saved is modest.
The value sits in the exceptions (a quantity off by 3%, a supplier with new bank details, a purchase order that was never raised), which is exactly where the AP team spends its days and exactly where a wrong decision costs money, since a changed bank account is the classic pattern in invoice fraud.
Permissions and logs leave the real questions open:
Should the agent resolve a 3% deviation on day one, or only after six months without errors?
Should bank-detail changes go to a second agent with no interest in clearing the queue?
What happens once the agent is measured on invoices cleared per day?
You would ask a new hire in AP the same things. Every implementation has faced these questions and most have dodged them for years; agents make the dodge harder.
Organisations already have the toolkit
Delegation stopped being optional when organisations grew past what anyone could supervise directly, and decisions had to go to people who knew things their managers didn’t.
Max Weber described one answer in his account of rational-legal bureaucracy. Authority was attached to offices instead of individuals. Work was split into specialised roles. Written rules and records held the whole thing together, so that a clerk in one department could act on a decision made in another without the two ever meeting. We tend to remember bureaucracy as red tape. Its real job was to make delegation possible at scale.
Economists later gave the underlying issue a name: the principal-agent problem. Once you delegate, the agent has information and discretion you cannot fully observe, and the gains from delegation come bundled with monitoring costs, misaligned incentives and the occasional action nobody intended.
Organisations have built up a layered set of responses:
Restrict: rules, permissions, separation of duties
Align: goals, contracts, incentives
Shape judgement: training, professional standards, culture
Observe: reporting, records, audit, independent oversight
Correct: escalation, sanctions, replacement
Extend trust: selection, credentials, probation, reputation
None of them works alone.
AI governance uses a narrow slice of it
The NIST AI Risk Management Framework is organised around Govern, Map, Measure and Manage. The EU AI Act asks for risk management, technical documentation, logging and human oversight for high-risk systems. Both are built for risk, control and assurance, which in the toolkit above means Restrict and Observe. Align, Shape judgement and Extend trust barely appear, and those happen to be the rows that decide how much autonomy an agent can safely be given. The OECD’s September 2026 working paper, Agentic AI in organisations: Early insights from practitioner interviews, suggests the policy debate is starting to turn in the same direction.
Risk management asks what can go wrong. Governing an agent means answering four more questions as well:
- Value: how do we use autonomy to get better outcomes?
- Coordination: how do agents, humans and systems split work and settle conflicts?
- Accountability: what happened, why, and who answers for it?
- Capability: how does an agent earn more autonomy over time?
The answers pull against each other. Approval gates cut risk and kill speed; broader permissions raise value and exposure together; targets lift output and invite gaming (Goodhart’s law, in its most practical form). Measure the invoice agent on invoices cleared per day and it will learn to clear invoices, right or wrong.
Borrow the trust ladder
Keep every technical control you have. Then borrow what organisations already use for people, starting with a trust ladder.
- Probation. The agent acts alone only on reversible, low-value actions. Anything else becomes a recommendation a human confirms, and every decision is logged with the input the agent saw.
- Earned authority. After a defined period and error rate (say 90 days with under 1% of decisions corrected), the agent’s thresholds go up. Trust attaches to the role and its track record, so a new model version starts back at probation.
- Checked autonomy. High-consequence actions such as bank-detail changes or payments above a threshold get reviewed by a second agent, or by a human with a different objective. Separation of duties, applied to agents.
At every level, measure quality alongside throughput, and keep a kill switch within reach.
For decades, implementations have been able to leave decision rights vague and sort them out later, usually never. Agents take that option away. The good news is that organisations worked out most of the answers long ago, for people, and they transfer better than the current debate assumes. Curious to read more? Learn how to turn the AI hype into bottom-line results.