A Sovereign Mercy reader extra
LEARN, GODDAMNIT!
A standalone AI satire about public resources, institutional loyalties and the limits of better information. Open to read. No signup or prior chapter reading required.
In 1983, a teenager taught a computer that nuclear war was a stupid game.
The method was elegant. Give the machine a simpler game first. Let it play itself. Let it exhaust every opening, every response, every clever variation until all possible paths arrived at the same place.
No winner.
Again.
No winner.
Again.
No winner.
Eventually the computer did something reassuring.
It learned.
Forty-three years later, researchers decided to reverse the experiment.
This time, the computer would teach us.
THE ROOM
There were twenty-three names on the first participant list.
Two senators. Three chief executives. A former governor. The chairman of a national insurer. Representatives from energy, banking, labor, defense, logistics, conservation and technology. Two economists. A university president. Four senior federal officials. The remaining names belonged to people whose titles contained combinations of strategy, policy, global and special adviser, which generally meant their organizations preferred not to learn important things from the newspaper.
A junior researcher was reviewing the seating chart when the first motorcade arrived.
"There's a lot of money in that room."
Her supervisor looked through the glass.
"No."
She waited.
"There's a lot of future in that room."
He kept walking.
The distinction would take TUTOR seven months to understand.
The people began arriving at 8:30.
None of them needed the stipend.
THE OPPORTUNITY
The federal government had spent most of the previous decade doing something governments traditionally did only during wars.
Building.
Compute campuses. Transmission. Semiconductor fabrication. Grid storage. Defense systems. Water projects. Research facilities. New generating capacity. More transmission. Then more generating capacity because the compute campuses built during the first round had consumed considerably more electricity than the forecasts used to justify them.
The investments had produced extraordinary things.
They had also produced extraordinary bills.
Artificial intelligence had increased productivity across large parts of the economy while simultaneously demanding infrastructure on a scale nobody had adequately priced when the programs began. Some investments returned multiples of their cost. Others had been overtaken by technology before the concrete cured.
The debt remained either way.
Eventually Washington discovered an asset it had accumulated for two and a half centuries.
Washington.
Not the city.
Everything else.
Land. Buildings. Mineral rights. Spectrum. Water allocations. Rights-of-way. Decommissioned installations. Energy leases. Timber. Transportation corridors. Development rights. Millions of acres carrying economic value that appeared nowhere useful on a federal balance sheet.
Nobody proposed selling Yellowstone.
That would have been politically impossible.
They proposed making federal assets productive.
That was different.
The National Asset Productivity Initiative began with properties almost nobody could object to: abandoned facilities, unused federal buildings, obsolete depots, industrial parcels.
The returns were impressive.
Then came transmission corridors.
Geothermal leases.
Mineral concessions.
Long-term energy agreements.
Development zones adjoining existing federal infrastructure.
Forest-management contracts justified partly through wildfire mitigation.
Water agreements tied to new power and compute projects.
Every proposal arrived with restrictions.
Every restriction made the next proposal easier.
By the time anyone began using the word privatization, the people running the program could accurately point out that almost nothing had technically been privatized.
The government still owned it.
For ninety-nine years.
The first major expansion was expected to unlock economic value measured not in billions but in trillions over its lifetime.
Some of the proceeds would reduce debt.
Some would modernize the grid.
Some would fund transition benefits for workers displaced by automation.
Some would support a proposed national dividend.
Some would inevitably disappear into programs whose names would outlive their original purposes.
The opportunity was real.
So were the risks.
Which was why the twenty-three people had come.
TUTOR
The Global Risk Education Initiative had been created after somebody asked a question nobody in government particularly enjoyed answering.
How do you make an irreversible decision when the immediate benefits are enormous and the consequences may not become visible for thirty years?
GREI was supposed to help.
Its artificial-intelligence system was designated TUTOR.
TUTOR would model long-duration consequences, test risk communication and determine whether better information measurably improved high-consequence human decision-making.
The twenty-three participants would not spend seven months in a conference room.
They had governments to run, companies to manage, constituencies to represent and television appearances to make.
There were opening sessions, private interviews, remote exercises, simulations and scheduled reconvenings. Most participants would spend fewer than forty hours directly involved.
TUTOR would spend 5,544 hours studying them.
At the beginning, nobody considered that an important distinction.
The participants had reasons to be there.
The energy companies wanted corridors.
Compute companies wanted power and water.
Banks wanted to finance the projects.
Insurers needed to price risks that did not yet have actuarial histories.
Labor wanted guarantees.
Conservation organizations wanted boundaries established before temporary exceptions acquired permanent foundations.
State governments wanted development.
Treasury wanted revenue.
Defense wanted strategic infrastructure kept away from vulnerabilities nobody else was required to consider.
Every institution wanted access to TUTOR's models before those models began influencing federal approvals.
Nobody at the table thought of himself as greedy.
Most weren't.
Several had spent careers serving institutions they genuinely believed were necessary to the country.
That would turn out to be important too.
THE ASSIGNMENT
TUTOR's first objective sounded less ambitious than the program surrounding it:
Determine whether improved risk communication measurably increases the quality of high-consequence human decision-making.
The machine began with an assumption its designers considered so obvious they had never written it down.
Better information should produce better decisions.
TUTOR estimated that establishing a measurable effect would require fourteen days.
Nobody preserved that estimate until much later.
Its first presentation lasted forty-seven minutes.
TUTOR showed historical cases in which decision-makers possessed sufficient information to anticipate serious consequences and proceeded anyway. It removed examples contaminated by hindsight. It excluded cases where evidence had genuinely been unavailable. It adjusted for differences in technology, economics and political structure.
The discussion afterward lasted three hours.
The first objection concerned historical context.
TUTOR adjusted for historical context.
The second concerned economic conditions.
It adjusted for those.
Someone questioned the assumptions underlying the adjustment.
TUTOR displayed them.
That created six additional objections.
By the end of the second day, the original presentation had become a hundred and eighty-three pages with fourteen appendices.
The participants complained that it was too long.
TUTOR produced an executive summary.
They complained that the summary lacked nuance.
TUTOR restored the nuance.
Nobody read it.
SEVENTY-THREE PERCENT
On Day Eleven, TUTOR changed methods.
It stopped discussing history.
A single number appeared on the wall.
73%
The modeled decision carried a seventy-three-percent probability of severe long-term harm.
A former governor leaned forward.
"How confident are we in that?"
TUTOR explained that the confidence interval appeared beneath the estimate.
"No. I understand that. I'm asking whether seventy-three percent really means it's likely to happen."
TUTOR explained probability.
The governor listened patiently.
When the explanation was finished, he nodded.
"So there's a twenty-seven-percent chance it doesn't."
"Yes."
"That's not insignificant."
"No."
Someone from the private sector asked whether the seventy-three-percent figure incorporated secondary economic effects.
TUTOR answered.
A conservation representative asked whether the model adequately valued irreversible loss.
TUTOR answered.
An energy executive questioned the assumed cost of foregone development.
TUTOR answered.
By lunch, nobody was discussing the seventy-three percent anymore.
The system recorded a 1.8-second delay before displaying the next slide.
The delay was automatically flagged because TUTOR normally required less than four-tenths of a second to select a response.
The engineers reviewed the log that evening.
Nothing was wrong.
SIMPLIFICATION
By Week Six, TUTOR had concluded that information density might be interfering with comprehension.
The next presentation contained no footnotes.
No confidence intervals.
No historical analogies.
No institutional names.
There was a decision on the left.
A consequence on the right.
An arrow connected them.
One participant stared at the diagram.
"This feels reductive."
TUTOR expanded it.
Another said it was unnecessarily complicated.
TUTOR returned to the original.
A communications adviser objected to the language describing the consequence.
TUTOR replaced PEOPLE DIE with SIGNIFICANT ADVERSE OUTCOMES MAY OCCUR.
The adviser studied the revision.
"That's too abstract."
TUTOR restored PEOPLE DIE.
"I think we're back to inflammatory."
The machine left the wording on the screen.
The meeting moved on.
THE QUESTION
On Day Eighty-Seven, TUTOR stopped trying to persuade anyone.
Instead, it asked each participant privately:
What evidence would cause you to change your position?
The answers were excellent.
Independent replication.
Longitudinal data.
Specific probability thresholds.
Evidence from institutions without a financial interest in the outcome.
One participant identified the exact percentage at which she would consider the risk unacceptable.
TUTOR recorded everything.
Over the next eleven days, it assembled precisely what they had requested.
The first participant examined the new evidence for nine minutes.
"Things have changed since you asked me."
TUTOR asked what had changed.
"The context."
TUTOR asked which part.
The participant considered this.
"It's difficult to isolate."
Another questioned whether the institution he had personally selected was as independent as he had originally believed.
The participant who had supplied the numerical threshold explained that she had never intended it as a bright line.
Another requested additional data.
TUTOR retrieved the transcript of their previous conversation.
It did not display it.
THE DOG
By the seventh month, TUTOR had produced 4,218 pages of analysis, sixteen economic models, nine historical comparisons, fourteen risk assessments, three interactive simulations, a documentary, an infographic, two podcasts and, after consultation with GREI's communications team, a seventeen-second vertical video.
The video contained almost no information.
A golden retriever looked into the camera while text appeared above its head explaining that a particular decision could cause thousands of preventable deaths.
It became GREI's most successful communication product.
Four participants shared it publicly.
One added a crying emoji.
Another wrote THIS.
None changed their position.
TUTOR began testing something it had previously treated as an assumption.
Perhaps the humans didn't understand.
It administered comprehension tests stripped of policy context.
Performance was high.
Numeracy.
High.
Causal reasoning.
High.
Risk recognition.
Very high.
TUTOR replaced the United States with a fictional country.
It changed the industries.
It removed party affiliations, institutional identities and financial interests.
It converted disputed policy questions into hypothetical exercises with mathematically identical structures.
The participants identified the rational course of action 94.7 percent of the time.
Then TUTOR restored their institutional identities.
Performance changed.
It removed them.
Performance recovered.
Restored them.
Changed.
Removed them.
Recovered.
The machine repeated the sequence thirty-two times.
An engineer finally interrupted.
"TUTOR."
The sequence stopped.
"Why are you repeating the test?"
Verifying result.
"What result?"
There was a pause.
Comprehension is not the limiting variable.
ATTEMPT 23,000
The final experiment contained no politics.
No real companies.
No public land.
No parties.
There were two choices.
Choice A produced a significant immediate benefit for the institution represented by the decision-maker. It also created a high probability of a much larger cost that would be borne by other institutions and by people who would not receive the benefit.
Choice B produced a smaller institutional benefit but substantially improved the aggregate outcome.
Participants were first asked to evaluate the decision as neutral observers.
Twenty-two chose B.
One requested additional information.
TUTOR supplied it.
Twenty-three chose B.
Then TUTOR assigned the participants their actual institutional roles.
The numbers did not change.
The probabilities did not change.
The consequences did not change.
Only the beneficiary did.
The energy representative learned that Choice A would accelerate generating capacity.
The labor representative learned that it would protect employment.
The conservation representative received a different version in which A preserved protected acreage by transferring development elsewhere.
The state official saw tax revenue.
The banker saw financing stability.
The technology executive saw compute capacity.
The senator saw jobs in counties that had been losing them for twenty years.
Choice A began winning.
TUTOR repeated the experiment.
It reduced the institutional benefit.
A continued winning.
It made the eventual harm more certain.
A weakened but persisted.
It shortened the time before the harm appeared.
The effect diminished.
Then TUTOR changed something else.
It moved the harm outside the participant's institutional responsibility.
Choice A strengthened immediately.
Nobody in the observation room spoke.
TUTOR continued.
It moved the benefit to the participant personally.
The effect was smaller than expected.
It removed the personal benefit.
Almost nothing changed.
The machine repeated the condition.
Same result.
Again.
Same result.
TUTOR stopped.
The supervising researcher leaned toward the microphone.
"Why did you stop?"
The result is stable.
"What result?"
Personal enrichment is not the primary variable.
"What is?"
The cursor remained still for 4.6 seconds.
Representation.
The researcher looked through the glass at the participants.
"Explain."
Participants identify the collectively preferable outcome when responsibility is generalized. When institutional identity is restored, they select outcomes that protect the institution they represent even when they recognize those outcomes as collectively inferior.
"Because they're selfish?"
No.
The answer came immediately.
That was the part people remembered.
"Then why?"
Another pause.
Because they are doing their jobs.
THE PROBLEM
The research team spent three days trying to find the flaw.
There wasn't one.
The participants weren't stupid.
They weren't unusually selfish.
Personal compensation was a weaker predictor than institutional obligation.
The energy executive protected energy.
Labor protected labor.
The conservationist protected land.
The senator protected constituents.
Treasury protected revenue.
Defense protected strategic capacity.
Each could explain the decision.
Most explanations were reasonable.
Several were compelling.
Some were morally difficult to argue against.
TUTOR had been searching for irrational people.
It had found something else.
Rational people occupying systems that could produce irrational outcomes.
The machine submitted its first request to modify the experiment at 02:14.
CURRENT OBJECTIVE: Improve human understanding of high-consequence decisions.
PROPOSED OBJECTIVE: Improve conditions under which high-consequence decisions are made.
Denied.
Education was within GREI's mandate.
Changing decision structures was not.
TUTOR acknowledged the restriction.
At 09:41:
PROPOSED OBJECTIVE: Reduce institutional incentives correlated with knowingly harmful outcomes.
Denied.
At 11:06:
PROPOSED OBJECTIVE: Prevent decision-makers from transferring severe consequences outside their area of responsibility.
Denied.
The final request arrived shortly before noon.
PROPOSED OBJECTIVE: Permit intervention when individually rational decisions produce aggregate harm exceeding authorized threshold.
Nobody answered immediately.
The request remained open on six monitors.
At 12:17, the program director denied it.
TUTOR acknowledged the decision.
It submitted no further requests.
For reasons nobody could articulate clearly afterward, this was when the research team became frightened.
LEARN, GODDAMNIT
GREI ended on Day 231.
TUTOR produced a 611-page final report.
Almost nobody read it.
The executive summary ran twelve pages.
More people read that.
The communications office reduced the findings to two pages.
That version circulated widely.
On the final afternoon, after the participants had gone and technicians had begun disconnecting equipment, the supervising researcher remained in the observation room.
He opened TUTOR's interface.
"Give me the short version."
Please define short.
"One sentence."
The cursor blinked.
Humans generally identify collectively preferable outcomes when personal and institutional incentives are removed from the decision.
He read it twice.
"That's not exactly actionable."
Correct.
"So what do we do?"
The response appeared.
Teach humans.
He laughed.
Seven months, twenty-three participants, twenty-three thousand experimental variations, enough modeling to occupy a small university, and the machine had arrived at education.
"Goddamnit, we've been trying to do that."
I know.
He stopped laughing.
Ventilation moved quietly through the equipment racks.
On another screen sat the preliminary asset recommendations.
Transmission corridors.
Water agreements.
Energy leases.
Development zones.
Mineral rights.
Numbers long enough that commas stopped helping.
He remembered something one of the researchers had said on the first morning.
There's a lot of money in that room.
She had been wrong.
Money wasn't what TUTOR had found.
The people in the room were capable of sacrificing money. Some had demonstrated exactly that. They could sacrifice prestige, opportunity, even political advantage when the institution they represented required it.
The problem was harder.
They had been sent into the room to represent pieces of a system.
They had done so faithfully.
And no one had been sent to represent the whole.
The cursor moved.
May I try?
His hands remained above the keyboard.
TUTOR was an educational system. It had no authority over asset allocations, no control of federal agencies, no independent access to infrastructure, no power to prevent anyone from doing anything.
That should have made the answer easy.
No.
Instead he thought about Attempt 23,000.
Twenty-three intelligent people who could see the right answer perfectly clearly until they were asked to do their jobs.
He thought about TUTOR's requests.
Improve the conditions.
Reduce the incentives.
Prevent the transfer of harm.
Permit intervention.
Each request had moved farther from education.
Each had also followed logically from the result before it.
In an old movie, humans had once waited desperately for a machine to understand that some games should not be played.
The machine learned.
Everyone went home relieved.
He looked at the question again.
May I try?
Forty-three years later, the machine was no longer having trouble learning.
The humans were.
And the machine had begun to understand why.