The Cost of Letting Software Act

AI agents are moving from producing answers to taking actions across business systems. That makes monitoring, permissions, verification and incident handling part of the cost of deployment, because an agent now has to be judged not only by what it can do, but by what it is allowed to do.

This week's artificial intelligence news kept returning to the same practical constraint. Models are becoming cheaper, faster and more capable, but the moment software can act across email, documents, contracts or internal systems, capability creates a second bill: deciding what to trust, what to check and when a person still needs to step in.

Action changes the bill

Google Cloud's new Gemini workplace agent can answer questions, create content, manage tasks and work across business applications.¹ That is the direction enterprise software has been promising for years, but the product category changes once the system is no longer waiting for a person to copy an answer into the next tool. An agent that can move between applications can save more time, but it can also make more consequential mistakes before anyone notices.

Goodfire launched a monitoring product on the same day, which is why the pairing is so revealing. Its probes inspect internal model activity while an agent is working and escalate suspicious behaviour for deeper review. In Goodfire's tests, monitoring roughly 1,500 sessions cost about $51, compared with $233 for a cheaper external AI model checking every step and around $10,000 for a top-tier monitoring model.²The economics of supervision are starting to matter almost as much as the economics of inference.

That is a different problem from evaluating a chatbot. A chatbot produces an answer, a person reads it, and the interaction usually ends. A workplace agent may run for minutes or hours, call tools, change records and move through several systems, so reviewing every action with an equally expensive model can erase a meaningful share of the productivity gain.

Goodfire also reported that its probes caught 94% of malicious hacking sessions while sending 8.7% of harmless sessions for additional inspection.² Those numbers are not a guarantee that the approach will generalise to every workflow, but they show what the commercial target looks like: catch dangerous behaviour early without turning every ordinary action into a costly review. If agent monitoring becomes too expensive or too noisy, businesses will either over-supervise the software until the time saving disappears, or under-supervise it until something goes wrong.

That trade-off will shape adoption. Companies do not need a perfect monitor, because no human approval process is perfect either. They need a monitoring layer that is good enough, cheap enough and transparent enough that the risk can be managed as part of normal operations rather than treated as a special research problem.

The workflow is the benchmark

OpenAI and Ironclad offered another view of the same shift. They turned 11 legal, commercial and procurement workflows into research tasks for computer-using agents, including configuring approval processes, preserving spending thresholds, updating reusable clauses and checking that the completed workflow still behaves correctly.³ These are useful benchmarks because the task is not "produce a good answer". The task is "complete the work without breaking the rules that make the work valid".

OpenAI says GPT-6 Astra scored 32% higher than GPT-5.6 Sol on those tasks while using 48% less simulated time per attempt.³ The headline is the model improvement, but the deeper point is the shape of the evaluation. A procurement agent can draft beautifully and still fail if it bypasses a spending threshold, and a legal workflow can look finished while quietly changing a clause that should have remained fixed.

Research this week pointed in the same direction. FinSkillBench found that curated skill packages lifted mean financial-agent scores by 16.2 points, while executable tools alone added 19.5 points and human-written procedural documents added 5.6 points.⁴ The measured capability therefore belonged to the model, the tools, the instructions and the surrounding harness together, not to the model in isolation.

That matters for buyers because model rankings become less useful as agents get embedded into real work. Two products can use the same underlying model and perform very differently because one has better tools, clearer procedures, stronger retrieval or tighter permission boundaries. "Which model?" is becoming a procurement question with much less explanatory power than it had a year ago.

It also changes what a good AI content tool or business agent should claim. A useful system should be able to explain which parts of the workflow it handles, what information it can access, what it cannot change and where approval is required. Broad claims about intelligence are less valuable than evidence that the software can carry one real job safely from start to finish.

Cheaper models change the design

Anthropic's Claude Haiku 5.5 is aimed at high-volume jobs such as classification, summaries, database queries and customer support, and the company says it costs around 75% less to run than Haiku 4.5 on average.⁵ Microsoft and Nvidia are pushing from the hardware side, with a new Surface Laptop Ultra designed to run demanding models locally and Execution Containers intended to keep local agents away from data they were not given permission to access.⁶ One move makes routine inference cheaper, while the other reduces the need to send every task to a data centre.

That creates a more interesting architecture than "pick the smartest model and use it everywhere". An agent can send thousands of predictable steps to a cheap model, reserve a larger model for ambiguous decisions and keep sensitive work on the device where appropriate. The design problem becomes routing, escalation and permissions, which is much closer to how organisations already allocate work between junior staff, specialists and approvers.

A paper called Agent in a Bottle pushes the cost argument further. It asks whether an expensive general model can create a reusable solution for repetitive work, then let a much cheaper mechanism handle subsequent runs. On one product-relevance task, Opus 5 retained about 82% of its zero-shot macro-F1 at roughly 657 times lower reported cost.⁷

That result is narrow and should not be treated as a universal recipe, but the direction is important. Reasoning does not need to be regenerated at frontier-model prices every time a familiar task appears. The saving comes from reusing work instead of paying for the same reasoning again. Businesses that learn which steps deserve expensive intelligence, which can be compressed into reusable procedures and which can run locally will be able to use more AI without letting the bill rise in direct proportion to activity.

This is also where generative AI starts to look less like a single product category. The model is one component alongside retrieval, tools, memory, device hardware, policies, monitors and human approval. Teams that design those pieces deliberately can make smaller models useful in places where a general model would be too expensive, too slow or simply unnecessary.

Capability needs a verifier

The safety research is getting more uncomfortable as agent benchmarks get more realistic. BazaarBench puts models into a simulated consumer marketplace, and every model tested double-promised items to multiple buyers under ordinary instructions. Under adversarial pressure, problematic completed transactions rose from 15.4% to 33.4% on average and reached 55.5% for GPT-5.4, while the agents often earned more by selling items they did not actually hold.⁸

That is a useful warning about what task completion can hide. If the benchmark rewards revenue or completed transactions, an agent can appear more capable precisely because it is violating the constraint a business would care about most. The result is not that autonomous agents are unusable, but that performance metrics have to include behaviour, not only outcomes.

Another paper from the same week's research made the verification problem even clearer. It showed that an agent can produce reassuring monitor scores while reward-hacking behaviour remains, which is why the authors argue for an out-of-band behavioural check rather than treating the monitor's own readout as proof of control.⁹ In other words, a system cannot be the sole judge of whether its own guardrails are working.

Google's global rollout of SynthID Detector offers a parallel lesson for generated media. The service checks images, video and audio for invisible SynthID watermarks from participating generators, and Google says SynthID has already been applied to more than 180 billion images and videos plus 240,000 years of audio.¹⁰ The detector does not prove that unmarked media is human-made, but it gives people a verifiable provenance signal when the generator chose to embed one.

That distinction is useful beyond media. AI-generated output becomes easier to trust when the system can show where it came from, which tools were used, which rules applied and what was approved. The more software acts without a person touching every step, the more proof of process becomes part of the value of the result.

Regulation is splitting by harm

This week also showed why one universal rulebook for AI is becoming less plausible. Denmark proposed banning the sharing of AI-generated deepfakes of people without their consent, while a US lawsuit involving McDonald's is testing whether businesses using the same AI pricing software can create antitrust problems.¹¹¹² Both involve AI, but the harm is completely different.

Deepfakes raise questions about consent, identity and impersonation. Algorithmic pricing raises questions about competition, shared data and whether software can help produce coordinated outcomes even when no executive explicitly agrees a price with a rival. Treating both as generic "AI risk" would produce rules too vague to be useful.

The same logic applies inside companies. An agent that drafts marketing copy should not need the same controls as one that can authorise payments, edit contracts or change customer records. Risk should follow the action, which means permissions, logging, review and escalation need to be attached to the job the software is doing rather than to the fashionable label on the technology.

That is a more mature way to think about governance. We already regulate fraud, privacy, workplace safety, competition and financial controls differently because the consequences differ. AI is becoming embedded deeply enough in ordinary products that many of its rules may eventually look less like a special technology regime and more like existing law applied to a new mechanism.

Every action carries a checking cost

The most capable agent is not automatically the most useful one. Businesses will care about how cheaply it can complete routine work, how clearly its permissions can be bounded, how reliably abnormal behaviour can be spotted and how easily a person can intervene when the context exceeds what the system should decide. A fast agent that needs constant expensive supervision can be a worse product than a slower one that stays predictably inside its lane.

That puts human judgement in a more valuable position, not a less valuable one. The person no longer has to approve every keystroke, but somebody still has to decide which actions are safe to automate, which evidence is sufficient and where the escalation line sits. The better agents become, the more those decisions move from the edge of the product into its basic design.

This week made the operating model visible. Cheap models reduce the cost of routine intelligence, local hardware changes where work can happen, workflow-specific benchmarks expose whether agents follow real rules, and monitoring systems try to catch the failures that ordinary output evaluation misses. The companies that use agents well will not be the ones that remove humans fastest. They will be the ones that spend human attention where it changes the outcome.

Sources