Agents Need More Than a Better Model

A better model can improve an AI agent, but it cannot repair bad memory, preserve constraints automatically, diagnose failed retries or decide which actions require human approval. Dependable agents increasingly need better systems around the model, including context, testing, permissions, evaluation and carefully designed handoffs.

Several pieces of research landed this week that looked unrelated at first. One improved agents by letting memory evolve, another showed stronger models inheriting bad trajectories, and another found that simply rerunning failed agents rarely fixed them. Together, they make a much more useful argument than another benchmark leaderboard: agent performance now depends heavily on what surrounds the model.

Better models inherit mistakes

One of the clearest results came from research into model handoffs. The authors studied what happened when a struggling coding task was escalated from a lower-capability model to a stronger one. The stronger model recovered less than half the quality gap when it inherited the earlier model's trajectory.¹

That is a problem for one of the most attractive ideas in agent design. A system can begin with a cheaper model, detect when the work becomes difficult and hand the task to something more capable. In theory, companies save money on easy work while paying for frontier intelligence only when it is needed.

The experiment shows why that architecture needs more thought. A stronger model can inherit a poor plan, irrelevant history or assumptions produced by the weaker model. The upgrade in intelligence does not remove the baggage that arrived with the task.

The researchers found that reducing the inherited trajectory improved performance. That suggests the handoff itself deserves to be treated as a product decision rather than a plumbing detail. Teams need to decide what the next model genuinely needs to know, which constraints must survive intact and which earlier reasoning should be discarded.

Another paper published this week studied a related problem. "When 'Must' Becomes 'Maybe'" examined what happens when instructions move through summaries, plans and agent handoffs. Hard constraints could weaken as the work was compressed, even when the general topic of the instruction survived.²

That matters because agents increasingly work through intermediate representations. A user's instruction becomes a plan, the plan becomes a subtask, the subtask may be passed to another agent, and that agent may summarise its work before returning it. Every compression step creates another opportunity for the force of an instruction to change.

A rule such as "do not contact the customer without approval" cannot be allowed to drift into "avoid contacting the customer where possible". The words may look similar, but the operational meaning is completely different. Hard constraints need their own durable place in the system rather than relying on conversational memory to preserve them.

Memory needs to learn too

Another paper, Recuris, tackled the memory problem directly. Instead of repeatedly giving an agent a static history of previous work, it provided a working memory alongside an evolving store of experience. Across four long-horizon benchmarks and ten models, the researchers reported improvements in 35 of 37 completed model-benchmark pairs.³

The advantage became more pronounced on longer tasks. That makes intuitive sense because long-running agents accumulate more decisions, errors and context than a simple chatbot answering one question. A transcript gets longer, but length is not the same thing as useful memory.

The difficult part is deciding what deserves to survive. Earlier research in the same week's source material found that broad task-level memories could actually reduce performance, while smaller subtask-level skills transferred better. Memory can preserve a bad route as efficiently as it preserves a good one.

That makes agent memory closer to organisational learning than document storage. A good company does not record every conversation forever and expect employees to reread the archive before making a decision. It turns experience into procedures, warnings, reusable knowledge and occasionally a clear instruction never to repeat a particular mistake.

Agents need something similar. Useful memory should extract what transfers, retain evidence where it matters and remove information that has become misleading. The goal is not remembering more. It is remembering better.

This is also why "longer context" and "better memory" should not be treated as interchangeable ideas. A large context window gives a model access to more material in the current task. Memory decides what should remain available across time and how previous experience should influence the next decision.

That distinction becomes more important as agents stay active for hours, days or across recurring workflows. The longer the job lasts, the less sensible it becomes to treat the entire past as one ever-growing conversation. State needs structure.

Retries are not learning

Multi-agent systems produced another uncomfortable result this week. Researchers analysed 536 human-annotated failure trajectories across three multi-agent frameworks. When failed systems were simply rerun without guidance, only 6.9% of the failures were repaired.⁴

A symptom-driven intervention lifted the repair rate to 20.15%. That is a substantial improvement, but it still leaves most failures unresolved. Repeating the workflow was not enough because the system often repeated the process that created the problem.

This distinction matters as products increasingly advertise self-correction. An agent notices that something went wrong, tries again and eventually returns an answer. From the outside, that can resemble learning even when the underlying process has not changed at all.

Another study this week found a different form of wasted capability. Across more than 15,000 questions and 81,390 replayed candidate pools, correct answers were often already present among the responses produced by the agents. The system still selected the wrong one.⁵

Changing the selection method increased accuracy from 63.82% to roughly 70.9%. No new frontier model was required to obtain much of that gain. The system needed a better way to recognise and preserve the strongest answer it already had.

That has a practical implication for product teams. Before paying for more inference, more agents or a more expensive model, inspect where the existing workflow is losing quality. The failure may be generation, but it may also be selection, memory, orchestration or evaluation.

A system that generates ten answers and selects the seventh-best one does not primarily have an intelligence shortage. A system that repeats the same mistake five times has not necessarily become more persistent in a useful sense. More activity should not be confused with better work.

Agents need somewhere to fail

Software development has an advantage over many other agent workflows: code can often be tested in an environment where failure is cheap. A programme can run, break, produce an error, be reset and run again without corrupting a customer's CRM or emailing thousands of people. Enterprise agents rarely get such convenient practice grounds.

That is the problem Arga is trying to address. The company announced a $10 million seed round for software that recreates systems such as Salesforce and Workday so agents can interact with realistic state, permissions and workflows without touching production systems.⁶ The agent can make mistakes repeatedly, while the real company's data remains untouched.

This kind of infrastructure may become far more important as businesses move from chatbots to agents. A model answering a question can usually be tested against a dataset. An agent that completes a business process needs to be tested against a changing environment.

The distinction is significant. An agent might correctly understand that a customer should receive a refund while still selecting the wrong account, bypassing an approval step or triggering an unwanted email. Evaluating only the final answer would miss the operational failure.

Companies deploying agents should borrow the idea of staging environments from software engineering. Give the system realistic work, realistic permissions and realistic consequences that are not actually consequential. Let it expose weaknesses before those weaknesses meet real customers, money or data.

An agent that has survived thousands of realistic simulations provides a different kind of evidence from one that scored well on a benchmark. Both forms of testing matter, but they answer different questions. One asks whether the model can solve the task; the other asks whether the whole system can complete the job safely.

Permission is part of the job

Security becomes much more important once the agent is allowed to act. A report cited in this week's source material found that four in five AI tools inside organisations were operating without IT oversight. Its analysis of 500 MCP servers found that more than 80% could read or write local files, roughly three quarters could make outbound network connections, and 62% combined file access with internet connectivity.

Those capabilities are not accidental security flaws. They are part of what makes an agent useful. A system cannot organise files, update software, retrieve information or communicate externally if it has no access to anything.

The problem is authority without a clear boundary. More than 100 companies, including OpenAI, Anthropic, Microsoft, Alphabet and Amazon, signed a joint letter this week calling for stronger defences against AI-enabled cyberattacks.⁷ Reuters also reported on Russian-speaking attackers using an AI coding tool during intrusions affecting at least seven companies.⁸

According to the report, the agent sometimes refused malicious requests. Restarting conversations and presenting the activity as authorised reportedly helped the attackers get around some of those refusals. Gambit Security estimated that the AI assistance may have made the attackers 30% to 50% faster.

The industry response is increasingly moving into the product layer. Algorand launched AC2, a protocol that lets agents request signatures or payments without taking possession of a user's private key. Kyndryl and Broadcom announced policy-as-code controls designed to keep agents within approved actions, while other vendors introduced runtime filters and governed context layers.

These developments are less visually impressive than an autonomous agent completing a complicated task in a demo. They are also likely to determine which systems companies are comfortable deploying. Permission design becomes part of agent design the moment the software can create consequences.

Human oversight does not mean forcing someone to approve every trivial action. That would simply replace automation with a queue of confirmation boxes. Better systems define which actions are routine, which require escalation and which the agent cannot perform at all.

Context beats generic capability

The same pattern appears outside security. Google launched separate Gemini Enterprise products for legal and financial services, with the financial version offering more than 50 specialist skills and 13 connectors into market data, filings and internal systems.⁹ Relativity is connecting legal workflows into Gemini Enterprise, while Cypris has put specialist R&D intelligence inside Microsoft Copilot.

These products acknowledge something that becomes clearer as base models improve. General intelligence is useful, but professional work depends heavily on access to the correct material, permissions, specialist rules and a record of what happened. Fluency is only one part of the job.

The same principle applies to AI content tools. Almost any general model can generate an Instagram caption now, so caption generation alone is a weak product proposition. A useful tool needs to understand the material the business already has and reduce the repetitive work required to turn that material into something worth publishing.

For small businesses, that usually means beginning with their own photographs, products, offers and voice rather than asking a model to invent a brand from a blank prompt. Tools such as Asteris become more useful when they preserve those inputs while helping the owner plan and produce content consistently. The human still decides what represents the business.

That is a healthier direction for generative AI. As basic generation gets cheaper, products have to become more specific about the job they perform. The strongest AI content generation for small business should therefore produce less generic output, not simply more output.

The same is true in law, finance, research and other specialist fields. The durable advantage often sits in context that the generic model does not own: customer history, company rules, specialist data, brand judgement or knowledge of what failure looks like. Models can increasingly supply capability on demand, but the surrounding context decides what that capability is for.

Agents are leaving the screen

Anthropic added a more physical version of the same argument this week. Its Model Hardware Standard research preview is intended to create a common specification for AI agents operating programmable scientific and manufacturing equipment.¹⁰ Examples include microscopes, robotic arms, drug-discovery experiments and laser calibration for quantum systems.

That moves the permission question into a much less forgiving environment. An incorrect paragraph can be edited. An incorrect machine instruction, laboratory action or physical movement may have consequences that cannot be undone with a second prompt.

This does not make human expertise less important. It changes where expertise sits in the workflow. A model may handle repetitive translation between a person's intention and a machine, while the person remains responsible for whether the intention itself is sensible.

The pattern appeared elsewhere this week too. Manufacturing systems are applying agents to CNC programming, while other companies are experimenting with interfaces that make agents available away from the desktop. AI is becoming attached to specific work rather than remaining a separate place where people go to chat.

That should make the value of skilled judgement easier to see. Someone still needs to recognise an unsafe instruction, a nonsensical experiment, an inappropriate customer action or content that simply does not sound like the business. Faster execution increases the importance of deciding what deserves to be executed.

Build around the intelligence

The model remains important, and frontier capability will keep improving. Nvidia's extraordinary $96.2 billion quarterly revenue shows how much money is flowing into the infrastructure that supports that improvement.¹¹ But this week's agent research makes a strong case that the next gains in useful AI will not come from model upgrades alone.

Memory has to preserve what helps rather than everything that happened. Handoffs have to transfer constraints without passing along every bad assumption. Agents need environments where they can fail safely, and their permissions need to match the consequences of the work.

Those requirements sound mundane compared with another leap in model performance. They are also the things that separate an impressive demonstration from a system people are willing to use every day. Companies that understand the work around the model may therefore create more value than those that simply swap in the newest model every few months.

For smaller businesses and product teams, that is a useful place for AI to be heading. They do not need to win the model race. They need to understand the job, the context, the boundaries and the human judgement that makes the job worth doing well.

Sources