The AI Hotline Is Already Late
The Short Version
The United States wants an AI emergency line with China.
Good.
It may already be late.
On Sunday, Treasury Secretary Scott Bessent said the U.S. had proposed a new notification mechanism for AI incidents that rise to the level of national security. The proposal came during talks with Chinese Vice Premier He Lifeng, ahead of a meeting between Donald Trump and Xi Jinping this week.
The wording matters.
The U.S. proposed a mechanism.
China's official readout confirms that the two sides held a dialogue on AI, but it does not mention the notification proposal, its scope, or an agreement to adopt it.
So there is no hotline yet.
There is an idea for one.
And it arrived days after CNN reported that an AI-assisted U.S. intelligence report falsely identified cargo on a Chinese vessel as components of a nuclear weapons program. CNN, citing four sources familiar with the episode, said the military prepared to intercept the ship, armed personnel were getting ready to board it, and aircraft were already in the air before officials examined the report more closely.
CNN says the Pentagon and U.S. Special Operations Command Pacific did not comment. A Chinese Foreign Ministry spokesperson said he was not aware of the episode.
Treat the account accordingly: it is serious reporting based on anonymous sources, not a public military after-action report.
But if the core account is right, the most important AI safety story of the week is not a model escaping a sandbox.
It is a human command system nearly acting on an AI-shaped mistake.
Very reassuring. The emergency line is being proposed after the emergency rehearsal.
The chatbot did not make the decision
That sentence will become the first defense every time this happens.
Technically, it is true.
According to CNN, an analyst asked a chatbot about intelligence concerning the ship's manifest. The system combined open-source material with classified signals intelligence and reached the wrong conclusion. The analyst then used AI again to package the result into a standard intelligence report, which was circulated through the military.
A human wrote the query.
A human disseminated the report.
Humans prepared the operation.
There were humans in the loop everywhere.
And that is exactly the problem.
"Human in the loop" is not a safety system if the human cannot see which claims came from the model, which sources support them, how uncertain the system was, whether the model fused incompatible evidence, or whether the polished final document preserves any of that provenance.
The AI did not press a launch button.
It may have done something more ordinary and more dangerous: turned a weak inference into an official-looking artifact that moved faster than its evidence.
The report format carried authority.
The underlying claim did not deserve it.
A hotline only catches the last failure
Bessent's proposal is still useful.
The two largest AI powers need a trusted way to contact each other when an AI incident affects critical infrastructure, cyber operations, biological security, military systems, or the integrity of strategic warning.
AP reports that possible scenarios include AI-enabled cyberattacks, biological misuse, major model failures, and loss of human control. Those categories are broad enough to open a conversation and vague enough to cause a diplomatic argument during the incident itself.
What qualifies as an AI incident?
Does an incorrect intelligence assessment count if a person approved it?
Does a cyber operation count if AI generated only part of the exploit?
How certain must one government be before notifying the other?
How much evidence can it share without exposing intelligence sources?
Who receives the alert at 3 a.m.?
What stops the channel from becoming a tool for signaling, delay, or disinformation?
The OECD's common reporting framework offers 29 criteria for describing incidents across jurisdictions and sectors. That is useful groundwork. It is not a national-security protocol.
A bilateral mechanism needs a smaller, harder grammar:
- authenticated contacts on both sides
- clear severity thresholds
- a minimum evidence package
- protected handling for sensitive information
- a deadline for acknowledgment
- escalation paths when facts are disputed
- regular exercises to prove the channel works
- a post-incident process that separates notification from blame
Without those pieces, the hotline is diplomacy-shaped furniture.
Visible.
Reassuring.
Not load-bearing.
The first alert has to happen inside the institution
International notification is the third layer, not the first.
Before Washington can warn Beijing about an AI-related incident, the U.S. government has to know that an AI system materially shaped the event. CNN's account suggests that recognition came late because AI was used both to reach the conclusion and to package it into a familiar report.
That is a provenance failure.
Every high-stakes AI-assisted assessment should carry a machine-readable and human-visible record of:
- which model and version were used
- which prompts or analytical tasks it received
- which sources it accessed
- which claims were generated or transformed by AI
- which analyst verified each consequential claim
- which contradictory evidence was considered
- what confidence level survived human review
- who authorized operational use
This is not about stamping "made with AI" on classified documents.
It is about preventing a chain of command from mistaking fluent synthesis for independently verified intelligence.
The U.S. Department of War's AI Acceleration Strategy aims to put leading models into the hands of millions of military and civilian personnel across classification levels. Acceleration without a common provenance and verification standard means every unit can move faster toward a different definition of safe.
That is not distributed innovation.
It is distributed ambiguity with operational authority.
What to watch after the summit
Trump and Xi are expected to meet on Thursday.
The useful outcome is not a sentence saying both countries support safe AI.
We have enough sentences like that.
Look for operational details:
- Did both governments agree to the mechanism, or did they only agree to keep talking?
- Which agencies own it?
- Which incidents trigger notification?
- Does it cover military AI and intelligence failures, or only civilian model incidents?
- Will the countries test the channel before a real crisis?
- Can either side publish a post-incident account?
- Is there a shared vocabulary for evidence, uncertainty, and severity?
A hotline cannot prevent every mistake.
It cannot repair a bad model, a rushed analyst, a broken verification process, or a leader who wants to believe the alarming version.
But it can create one more pause before an AI-shaped error becomes a geopolitical fact.
That pause matters.
It just has to exist before the planes are in the air.