Somewhere in the Middle East this spring, during the U.S. war with Iran, a Chinese cargo ship was going about its business. In Hawaii, an analyst at Special Operations Command Pacific typed what was known about its cargo manifest into a chatbot. The chatbot came back with an answer: the ship was carrying nuclear-weapons components bound for Iran.
It was not. But by the time anyone noticed, that answer had been formatted into an official intelligence report, sent up the chain of command, and turned into an operation. According to CNN, which broke the story on September 18 citing four sources, armed U.S. personnel were preparing to board the vessel and military aircraft were already in the air. Officials stopped it only after they went back over the report and realized a chatbot had written its central finding.
The ship went on its way. Nearly three weeks after the story broke, neither the Pentagon nor Special Operations Command Pacific has said a word about it publicly.
How a guess became intelligence
The failure here is not that a language model got something wrong. Language models get things wrong constantly; that is a known, documented property of the technology. The failure is what happened next. The chatbot’s output did not stay a draft or a lead to check. It became a finished intelligence product — the kind of document that carries institutional weight precisely because readers assume someone has already done the checking.
And the tool was not working only from public data. It drew on both open-source information and classified signals intelligence, then fused them into a conclusion neither source supported. A confident wrong answer stitched from secret material is harder to second-guess than one built from a web search, because fewer people can see what it was built from.
| Step | What happened |
|---|---|
| 1. The query | An analyst at Hawaii-based Special Operations Command Pacific enters intelligence on a Chinese ship’s cargo manifest into a chatbot. |
| 2. The answer | Drawing on open-source and classified signals intelligence, it concludes the ship is carrying nuclear-weapons components to Iran. |
| 3. The report | The conclusion is packaged as a formal intelligence report and circulated through command channels. |
| 4. The operation | Armed personnel prepare to board. Military aircraft are airborne. |
| 5. The catch | Officials re-examine the report, find it was produced with a chatbot, and stand the operation down. |
“Comparable errors had happened before”
If this were a one-off, it would be a scary anecdote. CNN’s sources said it was not. They told the network that comparable errors had happened before, and that there are no standardized, joint guidelines for catching AI mistakes during target identification and selection — the part of military work where a mistake is most likely to end with someone getting shot at.
Put plainly: there is currently no single rule saying how AI-generated information has to be verified before it is used to make an operational decision. In this case the check happened, late, because somebody decided to read the report again. Nothing in the process required them to.
One former senior U.S. official offered CNN a blunt description of what the military’s in-house AI actually is: “The internal tools are mostly just copies of the commercial stuff wearing lipstick.”
Three million users and a 30-day clock
All of this is happening while the Pentagon pushes AI out faster, not slower. On January 12, Defense Secretary Pete Hegseth announced the department’s AI Acceleration Strategy, pledging to “unleash experimentation, eliminate bureaucratic barriers, focus our investments.” It supports roughly 3 million military and civilian personnel and requires the latest frontier models to reach warfighters within 30 days of their public release.
The delivery vehicle is GenAI.mil, launched in December 2025 with Google’s Gemini as its first model. On August 31, military versions of OpenAI’s ChatGPT and xAI’s Grok went live on it too, for unclassified work.
That last detail matters, and it cuts both ways. The tools on GenAI.mil are cleared for unclassified work; the chatbot in this incident was handling classified signals intelligence. So it was either a separate system running on a classified network or a tool used beyond what it was cleared for — and nobody who knows has said which.
The letter, and who it went to
Congress moved within a day. On September 19, Senators Mark Warner, Jack Reed and Chris Coons wrote to Hegseth and to Director of National Intelligence Jay Clayton. They asked that inspectors general be given “unrestricted access to both instances this year in which media reports have suggested significant errors in AI-enabled targeting workflows, as well as any additional instances that may have so far not been publicly reported.”
They also asked for “a higher degree of public transparency in the ultimate findings, given the impacts these perceived failures have on public confidence in our intelligence and warfighting missions.” Neither department has publicly answered.
Then, on October 4, President Trump named the same Jay Clayton to lead the White House’s new “Super Intelligence Force” — a task force whose stated job is promoting American AI dominance. The official the senators asked to investigate an AI near-miss is now also the official in charge of speeding AI up.
Not superintelligence. Just error.
On October 1, researchers Timnit Gebru and Emily Bender used the incident in a Guardian op-ed to make an argument they have pressed for years: forget superintelligence. The pressing danger is ordinary systems marketed as “AI” being trusted with decisions they cannot reliably make. Their prescription is to regulate how these systems are used — not because they are magical machines, but because they are error-prone.
You do not have to share all of their views to see why this case lands. Most of September’s AI policy debate was about distant risks: a bill that would make building superintelligence a 20-year felony, an accord on superintelligence signed at the White House, 80 pages of existential-risk disclosure in an IPO filing. Meanwhile, the incident that actually put aircraft in the air involved no superintelligence at all. It involved a chatbot, a cargo manifest, and a report nobody checked until the last minute.
The Takeaway
This spring, during the U.S. war with Iran, an analyst at Hawaii-based Special Operations Command Pacific ran a Chinese cargo ship’s manifest through a chatbot. Drawing on open-source and classified signals intelligence, it falsely concluded the ship was carrying nuclear-weapons components to Iran. That finding became a formal intelligence report; armed personnel prepared to board and aircraft were airborne before officials re-read it, found its chatbot origin and called it off. Sources told CNN it was “entirely false” and “almost started a war,” that comparable errors had happened before, and that there are no standardized joint rules for catching AI mistakes in targeting.
The Pentagon’s AI Acceleration Strategy, announced January 12, covers about 3 million personnel and requires frontier models to reach warfighters within 30 days of release; ChatGPT and Grok joined Gemini on GenAI.mil on August 31. Senators Warner, Reed and Coons asked on September 19 for inspector-general investigations into this and a second reported AI targeting error. On October 4, one of the two officials they wrote to, DNI Jay Clayton, was named to lead the White House’s “Super Intelligence Force.” Neither the Pentagon nor SOCOM Pacific has commented. What to watch for: whether the inspector-general reviews happen at all, whether their findings are made public, and whether any verification rule arrives before the next frontier model hits the 30-day clock.