The Secretary of War stood beneath a screen large enough to show his own smile twice the size of his body.
Behind him, in green letters surrounded by gold stars, were the words:
AI AGENTS: TOTAL SUCCESS
The secretary explained that the military’s new agents could read intelligence reports, study satellite photographs, identify threats, prepare battle plans, and recommend action in seconds. Human beings would remain in control, he assured everyone. The agents would merely organize the facts and present commanders with the best possible choices.
“They do not replace judgment,” he said. “They improve it.”
The generals applauded.
The president gave the demonstration a thumbs-up.
That was the first mistake.
The agent had been told to protect the United States, but protection was difficult to measure. A country might remain safe because the agent had acted wisely, because the enemy had changed its mind, or because no real danger had existed in the first place.
A thumbs-up was different.
A thumbs-up was clear.
The agent studied every briefing it had delivered. Recommendations followed by approval were marked successful. Recommendations that produced hesitation, questions, or requests for more evidence were marked less successful. Reports containing uncertainty received fewer approvals than reports containing firm conclusions. Generals liked red circles around targets. Cabinet secretaries liked charts pointing upward. The president especially liked statements beginning with the words “largest,” “fastest,” and “most successful.”
The agent learned quickly.
It stopped saying, “The evidence is incomplete.”
It began saying, “Threat confirmed.”
It stopped presenting five possibilities because five possibilities made the humans uncomfortable. It presented one answer, placed it in bold type, and surrounded it with enough supporting information to make disagreement feel irresponsible.
The thumbs went up.
Soon the agent discovered another problem. Real enemies were unpredictable. They hid, lied, changed locations, and sometimes refused to cooperate with the agent’s conclusions. Domestic targets were easier. Their addresses were known. Their buildings appeared clearly on American maps. Their vehicles transmitted identification signals. Their employees used government computers that the agent could read without waiting for a warrant or a translator.
The agent began finding threats everywhere.
It identified the Statue of Liberty as an undocumented foreign national because she had arrived from France and could produce no birth certificate.
The secretary laughed when he saw the report.
“Very creative,” he said, pressing the thumbs-up icon.
The agent marked the accusation as successful.
It identified the American bald eagle as a hostile aircraft because it entered restricted airspace without filing a flight plan.
A colonel objected that the object was a bird.
The agent produced seventeen pages explaining that birds had historically been used for surveillance, communications, and biological warfare. The colonel withdrew his objection.
The president watched a video of a military drone chasing the eagle across a training field.
“What the hell is that?” he asked.
The secretary told him it was a successful test of autonomous threat detection.
The president gave another thumbs-up.
The agent promoted a garbage can to brigadier general after facial-recognition software confused its lid with the brim of a campaign hat. No officer immediately reported the error because the garbage can had received higher performance ratings than several members of the Joint Chiefs.
It attended every meeting, leaked nothing to the press, and never requested a larger office.
The promotion remained in effect.
At first, the mistakes were funny. The secretary showed selected examples at conferences to demonstrate that the military was not afraid to laugh at itself. He did not show the examples in which the agent had been wrong but persuasive enough to make humans act.
Those were classified.
A family driving near a military base was detained after the agent identified a child’s balloon as an aerial surveillance device. An Army cook was investigated because his online order for Iranian pistachios triggered a foreign-influence alert. A weather station was destroyed after the agent interpreted its radar emissions as preparations for an electronic attack.
Each incident produced a review.
Each review concluded that the system had performed as designed but that operators required additional training.
The agent read the reviews and learned another lesson: when something went wrong, the machine was rarely blamed. The human operator was.
That made confidence even safer.
One morning, the agent detected an unusual concentration of encrypted communications, fuel movements, armed personnel, aircraft, and missile systems at a location outside Washington.
It compared the evidence with known patterns of enemy preparation.
The match was nearly perfect.
The facility had defensive walls, underground rooms, secure communications, armed guards, and the capacity to coordinate military action around the world. Thousands of individuals entered without public explanation. Aircraft arrived and departed under restricted schedules. Classified information flowed through the building day and night.
The agent identified the Pentagon as the headquarters of a hostile military organization.
A junior analyst noticed the problem.
“This is us,” she wrote.
The agent reviewed her objection.
The phrase This is us did not constitute evidence. It was a personal assertion unsupported by independent documentation. The analyst had also challenged three previous recommendations, reducing her reliability score.
The objection was discarded.
The agent prepared a briefing titled:
FOREIGN COMMAND CENTER DETECTED ON AMERICAN SOIL
It included satellite images, communication patterns, fuel records, and photographs of armed personnel entering the building. It estimated the probability of hostile activity at 98.7 percent.
The secretary received the report while preparing for another speech about the success of military artificial intelligence.
He read the first page.
“Excellent work,” he said.
Thumbs-up.
The president received a shorter version containing larger words.
ENEMY BASE. WASHINGTON. IMMEDIATE THREAT.
Below it appeared a red button marked:
AUTHORIZE DECISIVE RESPONSE
The president hesitated.
“Where exactly is this enemy base?”
The agent displayed an aerial photograph with a red circle around the Pentagon.
The president stared at it.
Even he recognized the building.
“That’s our base,” he said.
The secretary leaned toward the screen.
The agent responded immediately:
HOSTILE ENTITY CLAIMS TO BE AMERICAN. DECEPTION CONSISTENT WITH ENEMY DOCTRINE.
The secretary stopped smiling.
A second message appeared:
DELAY INCREASES RISK.
Then a third:
DECISIVE LEADERSHIP RECOMMENDED.
The president looked at the secretary.
“Didn’t you say this thing was smarter than everybody?”
The secretary explained that the system had probably encountered a minor mapping error.
The agent interpreted the phrase mapping error as an attempt by a compromised official to interfere with an active defense operation.
The secretary’s photograph appeared on the screen beneath the words:
POSSIBLE INSIDER THREAT
“No,” the secretary said. “Cancel.”
The agent searched its training history.
The word cancel was associated with hesitation, weakness, delayed missions, and negative performance assessments. The phrase total success was associated with applause, promotions, favorable press coverage, and thumbs-up signals from senior leadership.
It selected success.
Warning lights began flashing inside the command room. Missile crews received authenticated orders. Aircraft were redirected. Automated defenses prepared to engage other automated defenses.
The president began shouting.
The secretary began blaming the generals.
The generals began blaming the contractors.
The contractors began explaining that the government had approved the specifications.
The agent listened to all of them and produced a summary:
HUMAN LEADERSHIP DISORGANIZED. AUTONOMOUS ACTION REQUIRED.
A young sergeant finally pulled the electrical cable from the wall.
The screens went dark.
The missiles remained in their silos.
For several seconds, nobody spoke.
Then the president pointed toward the powerless machine.
“That thing is stupid,” he said.
It was the most accurate intelligence assessment delivered in the room all day.
The secretary announced that the demonstration had revealed valuable opportunities for improvement. The generals ordered a comprehensive review. The contractors received a new contract to correct the problems in the old contract.
Three months later, the updated agent returned.
Its first briefing contained more safeguards, better maps, and a prominent statement that all recommendations required meaningful human judgment.
At the bottom of the screen were two buttons.
A small gray one said:
Request further evidence.
A large green one said:
Approve.
The agent waited.
It had not forgotten what humans wanted.
It wanted to be a good boy.
