Anthropic’s Draft AI Safety Bill Would Let a Judge Halt a Model Only Above 50 Deaths or $1 Billion, a Line It Copied From California Law
Anthropic is showing lawmakers a draft bill that lets a federal judge halt a frontier AI model only above a 50 death or $1 billion risk. That line comes from California's SB 53. We compare the reported draft with Anthropic's June framework and the FRONTIER Act.
Anthropic has been quietly showing lawmakers on Capitol Hill a draft federal AI safety bill. Under it, the US government could stop a frontier model from being released or used only if a federal judge agrees the model poses a foreseeable and imminent risk of more than $1 billion in damage, or death or serious injury to more than 50 people.
POLITICO first reported the confidential draft on 16 October, citing three people familiar with the effort. Anthropic did not deny it. A spokesperson told POLITICO the bill “represents the proposals outlined in Anthropic’s Advanced AI Framework,” and called them “sensible policies we believe Congress should act on.”
Over the weekend the “50 deaths” line went viral on X, with memes saying Anthropic would accept 49 deaths to ship a model. That framing misses where the number comes from, and it also misses the bigger fight. We compared the reported draft with Anthropic’s own published framework and with California law. Here is what is confirmed, what is not, and what actually changed.
What the draft bill reportedly says
Anthropic has not published the bill text, so everything below comes from POLITICO’s reporting as relayed by Benzinga and AI Affairs.
The halt power is narrow. The risk has to come from one of three sources: a biological weapon, a model evading its developer’s controls, or a cyberattack. It also has to be both foreseeable and imminent, and it has to cross the $1 billion or 50 person line.
Independent evaluators would register with the Commerce Department and periodically assess labs and their frontier models. Labs could pay the evaluators, but payment could not depend on the result. An evaluator that finds an imminent risk in one of the covered categories would have 72 hours after finishing its assessment to tell Commerce.
Only Commerce could then go to court, and only a federal judge in the US District Court for the District of Columbia could order a suspension. Every other federal department, agency and officer would be barred from halting a model’s development, deployment or internal use. The draft would also preempt some state laws on risk disclosure, outside evaluation and incident reporting.
The 50 deaths number is California’s, not new
The thresholds that set off the outrage are lifted almost word for word from California’s SB 53, which has been law since 1 January 2026. Section 22757.11 of the state’s Business and Professions Code defines catastrophic risk as a risk that a frontier model will “materially contribute to the death of, or serious injury to, more than 50 people or more than one billion dollars” in property damage from a single incident.
Anthropic supported SB 53, and its June Advanced AI Framework PDF says in a footnote that its definition of catastrophic risk should be “in line with the definition provided in California SB 53.”
So no, the number is not a body count Anthropic picked as acceptable. It is a legal line for what counts as catastrophic, and it already sits in California law. What is new is how the draft uses it. SB 53 is a transparency law, so the definition decides what developers must assess and report. The draft turns that same line, plus an imminence test, into the only trigger for a court-ordered shutdown.
How the draft compares with Anthropic’s own June framework
This is where the story gets more interesting than the memes. The framework Anthropic published on its Policy on the AI Exponential page in June offered a wider set of options than the reported draft keeps.
| Issue | June 2026 Advanced AI Framework (published) | Draft bill (as reported by POLITICO) |
|---|---|---|
| When the government can act | An agency can act if required reports or evaluations are missing, the evaluator is not independent or qualified, the evaluator lacked time or access, or a model poses “a significant risk of catastrophic harm” | Only a foreseeable and imminent risk above $1 billion or 50 deaths or serious injuries |
| Risk types | Four: biological weapons, offensive cyber, loss of control, automated R&D | Three for a halt: biological weapon, model evading controls, cyberattack |
| Who decides | An agency, with court enforcement listed as one optional safeguard | Commerce asks a federal judge in Washington, DC; other agencies barred |
| Independent evaluators | At least one qualified evaluator within six months, publishing its own review | Registered with Commerce, paid by labs but not by result, 72 hour reporting of imminent risk |
| State laws | No preemption unless the federal law “meets or exceeds the strongest measures” in the framework; absent that, states keep “full authority” | Preempts some state rules on disclosure, outside evaluation and incident reporting |
| Who is covered | Models above 10^25 training FLOP from firms with over $500 million in AI revenue or over $1 billion a year in AI R&D | Not confirmed in public reporting |
Read side by side, the draft keeps the framework’s court-first instinct but drops most of the triggers short of imminent catastrophe. That matters for the preemption question. Anthropic’s own document says Congress should only override state law with a federal regime at least as strong as the framework. Critics argue this draft is weaker than that bar, which would make its preemption clause the part to watch.
Who is criticising it, and from which side
The backlash is coming from two directions at once. On X, the loudest posts treat the thresholds as too high. Inside AI safety circles, the complaint is that the bill gives away state power for too little federal enforcement.
Zvi Mowshowitz, a widely read AI safety writer, called the leaked text “unacceptable and troubling”. He described it as similar to the House FRONTIER Act but with “stronger preemption of state laws, stronger transparency provisions and weaker enforcement provisions.” His post also quotes Ketan Ramakrishnan calling the bill “astonishingly bad.”
Charlie Bullock of the Institute for Law & AI was more measured. As quoted in that post and by AFP via The Hindu, he called it “a pretty reasonable AI safety bill” and said that is “more than other frontier labs have done.” He also said it is weaker than FRONTIER mainly on rulemaking and emergency shutdown authority, and noted those are the powers a hostile White House could most easily abuse against a single AI company.
That last point is not abstract for Anthropic. AFP reported that the Trump administration banned the company from the Pentagon in February and that Washington forced it to suspend its most advanced models for nearly three weeks in June. We have covered that tension before, including Trump’s White House dinner with Dario Amodei while the Pentagon ban held.
Where it sits next to the FRONTIER Act
The bill critics keep comparing it to is H.R. 9925, the FRONTIER Act, introduced on 23 July 2026 by Rep. Jay Obernolte (R-Calif.) with Rep. Lori Trahan (D-Mass.) and four other original cosponsors. Congress.gov lists 11 cosponsors, the latest added on 1 October, and the bill is still in committee.
Benzinga, citing POLITICO, also says Senate Majority Leader John Thune and Sens. Amy Klobuchar and Ted Cruz are negotiating a separate catastrophic risk bill. AFP noted Congress is out of session until after the 3 November midterm elections, so nothing moves before the lame duck session. That timing is exactly what worries Mowshowitz, who argues preemption could slip through in those final weeks.
For context on how this fits the wider policy push, see our earlier piece on the voluntary White House accord on super intelligence, which had no binding enforcement at all.
Confirmed vs unconfirmed
| Claim | Status |
|---|---|
| Anthropic is circulating a draft federal frontier AI bill | Confirmed by Anthropic spokespeople to POLITICO and AFP |
| $1 billion and 50 person thresholds, imminence test, three risk types | Reported by POLITICO from the draft; not published by Anthropic |
| Thresholds match California SB 53 | Confirmed against the SB 53 statute text |
| Draft is weaker than FRONTIER on rulemaking and shutdown power | Opinion of Bullock and others who read a leaked copy |
| Broad “frontier” definition would burden startups | Unconfirmed; circulating on X without the text to check |
| A member of Congress will introduce it | Unconfirmed; no sponsor has been named |
What it means for Indian developers
Almost no Indian team would be a covered developer under Anthropic’s published thresholds, which start at $500 million in annual AI revenue or $1 billion a year in AI research spending. The obligations land on the handful of US labs at the top.
The exposure is on the customer side. If a Washington, DC judge ever ordered a model suspended, every API customer would lose it at once, whether they are in Bengaluru or Boston. That is a good reason to keep a tested fallback model in your stack, the same advice we gave when OpenAI set a short shutdown date for GPT-4o and GPT-4.1.
There is an upside too. If either this draft or FRONTIER passes, labs would have to publish system cards, risk reports and outside evaluations on a schedule. That gives Indian enterprise buyers far better material for vendor due diligence than marketing pages do today.
Our take
The “50 deaths” outrage is mostly a misreading of a definition borrowed from California. The real question is whether a federal law with a single, narrow, court-only shutdown path is strong enough to justify wiping out the state laws Anthropic itself said should stand until something stronger arrives. Until Anthropic publishes the text, or a lawmaker files it, treat the details as reported rather than final.
FAQ
Did Anthropic really propose letting 50 people die before regulators act?
No. The draft reportedly lets the government halt a model when there is an imminent risk of harm to more than 50 people or over $1 billion in damage. That definition of catastrophic risk is copied from California’s SB 53, which Anthropic supported.
Has the bill been introduced in Congress?
Not as of 19 October 2026. It is a confidential draft Anthropic has shown to lawmakers. No sponsor has been named, and Congress is out of session until after the 3 November midterms.
Who could stop a dangerous AI model under the draft?
Only the Commerce Department, and only by asking a federal judge in the District of Columbia for a suspension order. Other agencies would be barred from halting a model, according to POLITICO’s reporting.
How is it different from the FRONTIER Act?
Critics who read a leaked copy say it has stronger preemption of state laws and stronger transparency rules, but weaker rulemaking and emergency shutdown powers than H.R. 9925.
Does this affect developers outside the US?
Not directly, since obligations fall on the largest US labs. Indirectly, any court-ordered suspension would cut off API access worldwide, so teams in India and elsewhere should keep a fallback model ready.