Workday, Rivian and Banks Are Putting Limits on Agents

Workday, Rivian and financial regulators are all narrowing where AI agents can act alone. Workday keeps agents inside permissions, Rivian is considering third-party agents in cars, and the FSB wants stronger human oversight in finance. The common pattern is autonomy with explicit stopping points.

A lot of agent news still gets measured by what software can do without us. This week was more interesting for the opposite reason. Across workplace software, banking, cars and courts, organisations are starting to define the moments when an agent has done enough and a person needs to take responsibility.

Workday keeps the approval step

Workday made Sana its new home experience this week, giving employees a natural-language route into tasks such as requesting leave, resetting passwords and handling expenses.¹ The agents operate inside existing policies, permissions and approval processes, and Workday says people are brought back into the flow when judgement is required. That detail matters more than the convenience of asking for leave in a chat box because it shows where the product boundary has been drawn.

Abacum made a similar choice for finance teams. Its Modeler can take a natural-language request or an existing spreadsheet and build a live financial model against connected company data.² Finance users see the proposed structure before anything is saved, while calculations still run through Abacum's deterministic engine. The agent gets substantial work to do, but it does not silently replace the controls around the financial record.

DXC is applying the same logic to IT operations. The company says its OASIS platform has been measured across more than 500 use cases, reducing resolution time by 95% on average and cutting some filesystem incidents from 53 to 90 minutes down to 3 to 6 minutes.³ Sensitive actions can still stop for human approval. Speed is being granted selectively rather than universally.

That distinction is easy to miss when every product announcement is framed around capability. Drafting an action, recommending an action and committing an action are different things. A useful agent can move quickly through the first two while still requiring explicit authority for the third.

This is where the phrase "human in the loop" becomes too vague. A person who approves every minor step becomes a bottleneck, while a person who sees only the final result may have too little context to challenge it. The better design is to decide which actions are cheap to reverse, which change important records and which create commitments that are difficult to undo.

Cars make consequence physical

Rivian CEO RJ Scaringe said this week that the company's vehicles could eventually support AI agents from third parties such as Anthropic.⁴ That takes the same permission question out of an enterprise workflow and puts it into a machine people physically sit inside. The distance between a poor suggestion and a harmful action becomes much shorter.

Anthropic has already started describing similar limits in its own usage policy. For models doing longer and more independent work, high-risk recommendations in areas such as health, finance and essential services still require a qualified person who can review and change the recommendation.⁵ When Claude is connected to hardware capable of causing injury, an operator must be able to observe and stop it, and the equipment must reach a safe state if Claude disconnects.

Those requirements reveal something useful about autonomy. More capability does not remove the need for permission design. It makes permission design more specific because the cost of a mistake depends heavily on what the agent is connected to.

Summarising a meeting is easy to reverse. Drafting an email is easy to inspect. Changing a financial position, controlling physical equipment or taking an action inside a vehicle sits in a different category. The consequence of the action should determine the amount of autonomy, not the fact that the model is technically able to perform it.

Amazon's Bee devices point to the same tension in a lower-risk setting. Bee can work across conversations, email, calendars and Amazon shopping, turning remembered information into tasks and actions.⁶ That can remove a lot of repetitive coordination. It also means a personal assistant needs increasingly precise rules around what it can remember, what it can infer and which actions require confirmation.

The best experience may therefore feel less autonomous at the moments that matter most. A system that pauses before spending money or changing something consequential may look less impressive in a demo than one that completes everything automatically. In daily use, that pause can be the feature that makes people comfortable delegating more of the surrounding work.

Banks formalise the boundary

The Financial Stability Board issued final guidance this week calling for stronger human oversight of agentic AI in finance.⁷ Banking makes the boundary especially visible because the same software can move from analysing information to committing capital. A recommendation and a transaction may sit next to each other in the interface while carrying very different consequences.

SBS is approaching the problem through data boundaries as well as approvals. Its NVIDIA-powered AI Foundation is designed so banks can use generative and agentic AI without customer data leaving their own environment, and Scottish Building Society is already using the platform in production.⁸ The model matters, but so does the environment in which it is allowed to operate.

The same principle appeared repeatedly across this week's agent research. RATED tested tool retrieval when APIs fail and reported that its method cut faulty-tool selection from 86.2% to 2.9% under a simulated 30% API failure rate.⁹ Another study comparing combinations of models, protocols and agent frameworks found no universally best stack. Completion rates, efficiency, stability and task time moved depending on the surrounding technology choices.

OnTrack looked at failure from a different angle. It monitors an agent's trajectory while it runs and tries to detect when the work is heading towards failure. On the reported SWE-bench trajectories, its abort policy saved about 18% of compute that would otherwise have been spent on failing runs.¹⁰ Five of the six runs it stopped were in fact heading towards failure.

Taken together, these results weaken the idea that a more capable model automatically produces a more reliable agent. Reliability also depends on tools, permissions, failure detection, data access and the rules governing what happens when something goes wrong. The model is only one part of the operating behaviour.

That matters for businesses evaluating agentic products. Asking which model powers a tool is useful, but it is no longer enough. Teams also need to know what the agent can access, which actions it can complete without approval, what happens when a connected service fails, what gets logged and whether a person can reconstruct the sequence afterwards.

Courts need evidence too

OpenAI and Meta told a US judicial panel this week that synthetic audio and video are becoming difficult for people and verification tools to distinguish from authentic material.¹¹ Only 15 of 931 judges surveyed said they had encountered deepfake evidence. The small number is a reason to prepare before the problem becomes routine, not evidence that provenance can wait.

Courts cannot treat authenticity as a matter of whether an image or recording looks convincing. Evidence has to survive disagreement. That means being able to establish where a file came from, whether it was changed, who controlled it and whether the people represented in it consented to its creation or use.

This has consequences far beyond legal proceedings. Greenhouse says four in five job seekers now use AI in their search and 86% of US recruiters say at least half of the applications they receive are AI-written. Twinnin launched tools this week for licensing, consent and takedown management around AI use of a person's likeness. Different industries are running into the same requirement: something needs to establish who made an artefact and what happened to it afterwards.

Detection will remain part of that work, but it is a fragile final line of defence. If a sufficiently good synthetic asset can fool both a person and a detector, then the more useful evidence lives closer to the source. Original files, editing history, permissions and approvals become part of the trust record.

Anthropic made a related product choice ahead of the US midterms by adding official voting information to Claude.¹² Rather than relying entirely on generated responses for a high-consequence factual topic, the product can direct people towards an authoritative source. Sometimes good system design means deciding that the model should not be the final authority.

The UK's work on preparing for serious incidents involving autonomous AI agents follows the same direction. The government is considering how significant incidents and near misses should be identified and reported. Once organisations have to explain an incident after it happens, logs, permissions and escalation rules stop looking like administrative detail. They become part of the evidence.

Limits make delegation possible

The common reading of these developments could be that AI agents are being constrained just as they become more capable. The more useful interpretation is that clear limits are what make wider delegation possible.

A business is unlikely to hand meaningful work to software if every action has the same level of authority. The confidence to delegate grows when people know that routine work can proceed automatically while expensive, irreversible or safety-critical actions trigger a different process. Permission boundaries are therefore not simply restrictions on an agent. They are part of the product's value.

The same applies to AI content tools. AI content generation for small business can remove repetitive work such as drafting captions, adapting existing material or preparing a content calendar. The owner can benefit from that speed without handing over the final decision about what represents the business publicly. Automation works best when the boundaries match the consequence of the task.

This also changes the work left for people. As routine execution moves into software, human involvement becomes less frequent but more important. People set the boundaries, review exceptions, interpret ambiguous situations and own decisions that cannot sensibly be delegated.

There is a risk hidden inside that shift. Junior staff have often learned a profession by doing lower-risk work repeatedly before taking on higher-consequence decisions. If agents absorb much of that practice, companies need a deliberate way to teach the reasoning that used to be learned through repetition. Otherwise the future reviewer may be asked to challenge an agent without ever having developed the judgement needed to do it well.

That is why the useful version of human oversight is not a person clicking approve at the end of every workflow. It is a person who understands why the boundary exists, has enough information to challenge the system and has genuine authority to stop or change what happens next.

Workday, Rivian and financial regulators are arriving at this from different directions. The pattern is still the same. The more consequential the action, the less sensible unlimited autonomy becomes.

The next generation of agents will probably do more work with less human involvement. That does not require removing people from the process entirely. It requires being much more precise about the moments when software has reached the edge of its authority.

Sources