All notes
5 min read

The Off Switch

Last week a government pulled a frontier AI model off the market. Follow the reason down to its root and the offence was reading source code and pointing out the bugs.

On Friday evening, a federal agency decided that a computer reading source code and flagging the bugs in it amounted to a threat to national security. At 5:21pm Eastern, it put that decision in a letter.

The letter went to Anthropic. It ordered the company to suspend access to two of its models, Fable 5 and Mythos 5, for any foreign national anywhere in the world, its own employees included. Nobody can check the passport of every person typing into a chat box, so the order collapsed into a single outcome: both models went dark for everyone. Hundreds of millions of people shut out before the weekend had properly begun.

Governments have throttled chips and blocked exports for years. What none had done before Friday was reach past the hardware and switch off the model itself. That part is new, and worth pausing on.

There is a version of this story that runs on donors and timing and quiet collusion. I will leave it where it lies, because the documented version is stranger and needs none of it.

Anthropic published its own account of what the government objected to: a method for getting around Fable 5’s safeguards. The company examined it. It surfaced a handful of minor flaws, every one of them already known. And the method itself, the mechanism that moved a federal agency to emergency action over a weekend, was to ask the model to read a particular codebase and fix whatever was broken in it.

That is the capability they acted on: reading code and closing the gaps in it, the most ordinary thing anyone with an AI coding tool does on a Tuesday afternoon.

There is a reason a capability like that cannot be walled off. Finding the weak point in a lock is the same skill whether you are the burglar or the locksmith the family calls when they are shut out of their own house. Software runs on the same logic. The people who keep hospitals and banks and the power grid standing spend their days doing the exact thing the US government has now treated as a munition: reading code, hunting for the soft spots, sealing them before someone less friendly arrives. Anthropic said as much, and noted that the same ability already ships inside competing models, OpenAI’s GPT-5.5 among them. It has been out in the world for years. On Friday a government tried to outlaw the act of opening the door.

A smarter objection has been going around, and it deserves an answer, because for a moment it holds. Anthropic, the argument runs, spent months describing its own technology as dangerous. It called Mythos too powerful to hand out freely. So the government simply took the company at its word. Anthropic built the cage; the state only walked it inside.

It reads well, but it does not survive contact with what Anthropic actually shipped. The dangerous version never reached the public. Picture a manufacturer that builds a track car for speeds no public road allows, keeps it for a handful of vetted drivers, and sells everyone else the same car with a limiter fitted. The limited one is Fable 5, released to the world. The unrestricted engine is Mythos, handed only to a small group of approved cyber-defence partners. What the public got was the restrained model, the one Anthropic stress-tested for thousands of hours, alongside the US government and the UK’s own safety institute, before it shipped. You cannot accuse a company of arming the public when the whole design was built to refuse to. They held back the engine and sold the limiter, and it was the limiter the government pulled.

Fable 5 itself is almost beside the point. You may never use it. What should hold your attention is the rule the government has just written into being.

The flaw it pointed at was narrow, the kind every model on the market carries, the kind Anthropic says probably cannot be scrubbed out of any system built today. If a weakness that ordinary is grounds to tear a product away from hundreds of millions of users, only two readings remain. Either the same standard now applies to everyone, and the entire frontier seizes up, because every lab ships with weaknesses like this one. Or the standard is being aimed, and someone owes us an account of why it came down on this company, in this week.

The recent history does the government no favours. This is the same Trump administration that, in February, moved to lock Anthropic’s products out of federal agencies after the company refused to let the Pentagon use them for autonomous weapons or the surveillance of US citizens. A federal judge who reviewed that move called it retaliation. A company asks for firmer limits and shortly afterwards finds itself singled out, not once but twice. You do not have to believe in a plot to register the shape of it.

None of this says a government should be powerless. The state should absolutely be able to stop something truly dangerous from reaching the public. But it has to happen in daylight, with the evidence on the table, through a process the company can actually answer. Friday had none of that. There was a spoken description of a narrow flaw, a letter vague about its own reasoning, and an order that went off like a fire alarm and cleared the building before anyone could ask what was burning.

We have been here before, with the cast lightly changed. In the 1990s the same instinct treated strong encryption as a weapon and tried to hold it behind export controls. It collapsed, for the plainest reason available: you cannot fence in a capability that is already everywhere. The maths was loose, and it stayed loose. This is that fight again, with code-reading standing in for cryptography.

So a switch was installed this weekend, and the government now knows exactly where to find it. On a verbal say-so, with nothing published, no hearing and no warning, a product used by a real share of the planet can be turned off. This time the stated offence was reading code.

What I keep turning over is the next time a hand reaches for that switch, and whether the offence will look as harmless from the outside as this one does. I have a view on where it goes from here, and I suspect yours runs differently. I would rather argue about it than have you agree with me.

Sources

Every factual claim above is drawn from a primary source or contemporaneous reporting:

  1. Anthropic, Statement on the US government directive to suspend access to Fable 5 and Mythos 5 (12 June 2026). The order, the 5:21pm timing, the jailbreak method, the previously known minor flaws, GPT-5.5, and the hundreds-of-millions figure.
  2. Anthropic, Claude Fable 5 and Claude Mythos 5 (9 June 2026). Fable as the safeguarded public model, Mythos as the same model with safeguards lifted for vetted partners, and the red-teaming before release.
  3. Nick Wakeman, Judge blocks Pentagon’s Anthropic ban, calling it illegal retaliation, Defense One (27 March 2026). The February federal ban, the two refusals, and the court’s retaliation finding.