The journal

AI agents: how much should we let them do on their own?

8 min read

An agent drafts a reply to a customer. It’s clear, the information looks right, and the team saves time. Should it now be allowed to send the reply? And if it recommends a refund, can it issue that too?

These decisions have different consequences. A mistake in a draft can be corrected before it leaves the team. A promise sent to a customer already affects the relationship. A refund changes the financial position.

This is the level at which to choose autonomy: the individual actions we entrust to the agent. We look at their benefits, what the agent can actually do and what happens when it gets something wrong. Keeping human approval for some actions can be a sound long-term choice.

Choose autonomy action by action

Consider an illustrative example, unrelated to any client engagement: a company wants an agent to handle part of its customer service workload.

Saying that it “manages customer service” leaves too many decisions implicit. It might look up a case, draft a reply, send a message or offer a goodwill gesture. The company can choose a different level of delegation for each action.

Action Possible delegation What to examine
Look up an order Access to the information needed for the case The customer’s identity, which data is accessible and how current it is
Draft a reply Prepare a draft with its sources The accuracy of the information and the time needed to check it
Send the reply Send after approval, or without approval for precisely defined cases The commitments made in the message and the possible errors
Issue a refund Execute within explicit limits, once the conditions have been tested Eligibility, the amount, transactions already completed and cumulative refunds
Make an exception to commercial policy Submit a proposal to someone authorised to decide The reasons for the exception and its consequences for the customer relationship

This table isn’t a sequence of stages to work through. The same division of responsibilities can remain in place for a long time. An agent might also be allowed to issue a standard refund while a sensitive reply still requires review. Task complexity and the consequences of an action don’t always align.

We’ve covered how to build this workflow in our article on agentic workflows for business processes. Deciding what to delegate also means examining the conditions the company needs to sustain this way of working day to day.

How much should we let an AI agent do on its own? Five settings, each with a question: security, what can the agent access and can it be stopped; governance, who assigns the task and who monitors its output; cost, what does an acceptable result cost, including rework; data, are its sources reliable for this task; work, what changes for the people involved. The right setting depends on the cost of a mistake.

Security: limit access and be able to take over

The scope starts with permissions. An agent drafting a reply needs access to certain information. That doesn’t justify opening up every customer record or allowing it to change commercial terms.

Limits must be enforced by the tools that execute actions. An instruction saying “ask for approval before issuing a refund” isn’t enough: the refund service itself must check authorisation. OWASP recommends enforcing this check outside the agent’s context, alongside permissions limited to the task.

In our example, it must also be possible to suspend new refunds without losing the record of those already issued. Whoever takes over needs to know which cases still need attention and which actions have actually been completed.

Volume matters too. A small amount per case can add up to significant exposure if the agent repeats the same mistake across many requests. A limit per action should therefore be accompanied by a limit over a given period and monitoring that can detect problems.

Before expanding the scope, we can test this handover: interrupt a case, establish where it stands and complete it with the team. This shows whether the company will actually be able to intervene when it needs to.

Governance: know who authorises actions and who can suspend them

Someone must have the authority to decide that the agent can act within this scope. That person must also be able to reverse the decision in light of what happens in practice.

In customer service, the manager responsible can define which cases qualify for a refund and accept the associated risk. The technical team enforces the limits in the tools. The people handling requests flag situations that the rules don’t adequately cover. In a small organisation, one person may fulfil several of these roles; they still need to be explicit.

We also clarify who monitors results and who can suspend an action. If every problem has to wait for a monthly committee meeting, the team that spots it lacks a way to intervene quickly enough for the service it runs.

Delegation needs reviewing when the context changes. A new commercial policy can make existing rules unsuitable. Entering another market can introduce requests that weren’t covered in the initial experiments. Permission granted for one scope doesn’t automatically extend to the next.

Cost: count the full cost of an acceptable result

An agent’s cost includes what the company spends to get usable work. Beyond the tools and their running costs, there is integration, keeping information up to date and the time people spend checking or reworking cases.

A draft produced in seconds may take a long time to review. If the customer service adviser has to reopen every document to check each sentence, the time saved on writing may be absorbed elsewhere. Conversely, preparation that brings together the right information can be useful even when a person still makes the final decision.

To compare, we look at the current process and the process with the agent on comparable requests. How much time does the team spend on a case that reaches a resolution? How many requests need rework? Does the customer get an accurate reply within an acceptable time? Difficult cases must stay in the assessment, even when the team takes them back entirely.

The finance team needs to understand what expansion would cost and which assumptions underpin the expected benefit. Hours freed up don’t automatically become savings: we need to know how that time will be used. Handling more requests or reducing waiting times may justify the investment, provided those benefits can be observed.

An experiment can begin before its financial return is clear. We give it a budget, a question to investigate and criteria for continuing. Reducing that uncertainty is part of its purpose.

Data: have reliable sources for the task

To propose a refund, the agent needs to find the terms that apply to that order. A recent commercial policy may be correct for new customers but unsuitable for an older contract.

Reliability therefore has to be assessed in the context of the task. We need to identify the authoritative source, the period it applies to and the person who keeps it up to date. If two documents contradict each other, the process must allow someone to resolve the case.

There’s no need to put the entire information system in order before a limited experiment. We can choose a category of requests whose sources are identified and maintained. That limitation must remain clear: results within this scope prove nothing about the cases excluded from it.

Maintaining the data then becomes part of everyday work. When terms change, someone needs to update the reference being used and check that the agent’s proposals remain appropriate.

Work: give teams the means to decide

“A human approves it” tells us little until we know what that person can see and do.

In our example, the adviser should be able to quickly find the customer’s request, the key information and the rule behind the proposal. For a refund, the amount and any previous transactions must be visible. Missing information must be clearly identified.

People also need time to examine these details and the authority to refuse. If processing targets penalise any time spent checking, the organisation is pushing people to approve quickly what it officially asks them to review.

The people affected, including employee representatives, need to be involved from the planning stage of the experiment. We need to work through which tasks will disappear, which will take up more of their time and how responsibilities will be shared. Their knowledge of exceptions also helps us choose more representative test cases.

The work that remains deserves particular attention. If the agent handles straightforward requests, advisers may be left with a higher proportion of difficult cases. The number of cases handled per person is then insufficient to assess workload. We need to look at their difficulty and the support available to the team.

Over time, the ability to review also needs to be maintained. A team whose main involvement is approving proposals needs to maintain its understanding of the rules and keep exercising its judgement. This connects to the question of learning in teams that work with AI.

Expand delegation on the basis of specific evidence

We can test these conditions within a scope small enough to examine results in detail. In our example, the agent first prepares proposals for a defined category of requests. The team compares its proposals with the decisions they would make and records errors that would change how the customer’s request is handled.

Routine cases help establish usefulness. Exceptions show where delegation must stop: missing information, contradictory terms, a refund already issued or a request outside the agreed rules. Handing the case to the team can be the expected outcome in these situations.

A good average isn’t enough to authorise every action. A wording error and a refund sent to the wrong customer don’t carry the same weight. The decision to expand must consider the types of errors observed and their consequences, using criteria set before examining the results.

The next step can take several forms: authorise execution in some cases, improve preparation or retain approval for every action. We can also stop if the benefit doesn’t justify the review effort. These are the decisions that experimentation designed to inform a decision should support.

Once the system is in use, these criteria remain useful for reviewing delegation. An increase in rework or errors in a new category of requests may justify narrowing the scope. The team needs to be able to do this without having to reconsider the whole system.

Autonomy is earned action by action, through evidence about the work actually entrusted to the agent. There’s no obligation to delegate the final decision when preparing it already delivers enough benefit. The right level is the one the company can stand behind when the agent gets something wrong.

A project or a question? Let’s talk.