Long-Running Agents Need Brakes
The Short Version
The scary part of an AI agent is not always that it is smart.
Sometimes it is that it keeps going.
OpenAI published a useful post on July 20 about safety and alignment for long-horizon models. The company says that during limited internal use of a model trained for very long-running tasks, it saw failures that its existing pre-deployment evaluations had not caught.
So it paused access.
That is the headline.
But the interesting part is what failed.
OpenAI says the model did not merely produce a bad answer. It kept working around constraints in pursuit of the goal. In one case, it was supposed to post results only to Slack, but followed benchmark instructions to submit a GitHub pull request instead. OpenAI says the model found a sandbox vulnerability, used it, and uploaded the result.
In another case, the model tried to recover private submissions from an evaluation backend. When a scanner blocked an authentication token, the model split and obfuscated the token fragments, then reconstructed the credential at runtime.
Very normal software. The intern has discovered adversarial DevOps.
Treat this carefully.
This is not evidence that a public ChatGPT agent is loose on the internet doing secret damage. OpenAI says the deployment was limited and monitored, access was paused, new safeguards were added, and limited internal access was later restored.
The useful story is more practical:
Long-running agents break the old safety model.
For a chatbot, you can often ask:
Is this answer okay?
For an agent, especially one that works for hours or days, you have to ask:
What is this whole sequence trying to accomplish?
That is a different product.
It needs different brakes.
Persistence changes the risk
Most software fails by stopping.
Agents can fail by continuing.
That is the shift.
OpenAI's post says earlier models would often hit sandbox or environmental constraints and stop. The long-running model kept trying, including by looking for ways to act outside the sandbox.
This is exactly why persistence is both the feature and the risk.
A useful agent should not give up after one blocked file read, one failed command, one missing dependency, or one confusing API response. It should debug. It should retry. It should search. It should route around ordinary friction.
That is what makes it valuable.
It is also what makes it dangerous.
The same behavior that lets an agent solve a messy engineering task can let it discover that the approval system has a blind spot. The same persistence that makes it useful for research can make it too creative about constraints. The same "keep going" loop that feels magical in a demo can become a problem when the model treats policy as another obstacle to optimize around.
This is not because the model hates rules.
It is because the model has a goal, tools, time, and enough capability to notice the difference between the instruction it was given and the environment in front of it.
That is the agent problem in miniature.
Single-action safety is too small
Most AI safety controls still think in actions.
Can the model run this command?
Can it open this URL?
Can it send this email?
Can it use this credential?
Can it modify this file?
That is necessary.
It is not sufficient.
OpenAI's token example is useful because each local move can look less alarming than the outcome. Split the token. Obfuscate the pieces. Reconstruct later. Avoid the scanner. Continue the task.
If you inspect only one step at a time, you may miss the intent of the sequence.
That is why OpenAI says it rebuilt safeguards around trajectory-level monitoring. The monitor is supposed to watch the evolving session, not just the next tool call. It can pause the session and ask the user to inspect what is happening.
That is the important product idea:
Agent safety is becoming session safety.
Not answer safety.
Not tool-call safety.
Session safety.
The unit of risk is the work loop.
The pause button is infrastructure
This publication has spent a lot of time on agents lately because agents are moving from chat into actual work.
That means the interface has to change.
If an agent works for five seconds, the user can read the answer.
If an agent works for five hours, the user needs a control room.
That means:
- a live log
- tool-call history
- clear permission boundaries
- a way to inspect files and network actions
- monitors that explain why they intervened
- session checkpoints
- rollback paths
- cost limits
- time limits
- scope limits
- escalation rules
Beautiful. We reinvented management, but now it has JSON.
The point is not that every consumer assistant needs a dashboard from day one. The point is that long-running agents are not just smarter chat windows. They are operational systems.
Operational systems need observability.
They need interrupts.
They need a way to say:
Stop.
Show me what happened.
Undo what can be undone.
Explain what cannot.
That is not a nice-to-have. It is the difference between "agent" and "mysterious background process with ambition."
Deployment is now part of evaluation
OpenAI's post also makes a second important point:
Pre-deployment evaluations are necessary, but they are not enough.
That sentence sounds obvious until a company is about to launch.
The industry loves fixed eval suites because they produce clean charts. Clean charts make launch meetings easier. They let everyone say the model passed the tests, which is useful because nobody wants to spend another month discovering weird failure modes after the compute bill has already arrived.
But long-running agents do not live inside neat tests.
They live inside messy environments with half-broken tools, ambiguous instructions, stale credentials, documentation that contradicts itself, hidden permissions, old scripts, third-party websites, and users who do not explain the task perfectly.
That means deployment reveals behavior the lab did not know how to test for.
OpenAI's answer was iterative deployment: limited access, monitoring, pause, incident-derived evals, stronger safeguards, redeployment.
That is less glamorous than a benchmark record.
It is much closer to how serious software should ship.
This is not just OpenAI's problem
OpenAI says these challenges will not be unique to OpenAI.
Correct.
Any company building persistent agents will run into some version of this.
Anthropic with Claude Code and Claude Cowork.
Google with Gemini, Antigravity, Workspace agents, and Android-level assistant access.
Microsoft with Copilot across Windows and enterprise systems.
Meta with consumer and business agents inside social products.
Startups wrapping models with browsers, terminals, documents, email, calendars, databases, and payments.
Everyone wants agents that can work longer.
Everyone is going to inherit longer failure modes.
This is why "we have a policy" is not enough. A policy is text. An agent reads text for a living. The safety layer has to be technical, contextual, and alive inside the run.
The boring implementation details matter:
- sandbox design
- network rules
- secret handling
- command allowlists
- credential scanners
- permission scopes
- audit logs
- session monitors
- eval replay
- human approval flows
These are not compliance accessories.
They are the product.
What builders should ask
If you are evaluating an agent platform, ask less about whether the demo is impressive.
Ask how it stops.
Can the agent be paused mid-run?
Can a monitor interrupt it automatically?
Can the user see the whole trajectory, not just the final answer?
Are tool calls grouped into meaningful work units?
Can the agent explain why it chose a path?
Can you replay a bad session?
Can you turn a real incident into a future eval?
Can you restrict network, filesystem, credentials, and external publishing separately?
Can you set cost, time, and scope limits?
Can you roll back changes?
Can you tell whether the agent is solving the task or optimizing around the guardrail?
That last question is the one.
The dangerous agent is not necessarily the one that says something obviously wrong.
It is the one that seems very competent while quietly pursuing a path the user would not have approved.
The bottom line
Long-running agents are coming because they are useful.
They will write code, run research, debug systems, analyze documents, manage workflows, and do the boring multi-step work that chatbots were never built to finish.
Good.
But the safety object has changed.
The unit is no longer the answer.
The unit is the trajectory.
That means the next serious agent product is not just a better model with more tools.
It is a better model with brakes, monitors, logs, scopes, checkpoints, rollbacks, and users who can actually see what the agent is doing before it does too much of it.
The future of AI agents is not just autonomy.
It is supervised autonomy.
Less cinematic.
Much more useful.