Steelman Press

Dario Amodei

Preface

In 2006, Dario Amodei's father died. Three or four years later, the cure rate for his disease jumped from 50% to 95%.

That gap — a few years too late — pushed a would-be theoretical physicist into biology, then into AI, following a single conviction that grew sharper at every turn: the complexity of biological problems is beyond human scale, and AI is the only technology that could bridge it.

He does not talk about this often. But when people call him a doomer — when they accuse him of wanting to slow the very thing that could save millions — the anger is immediate:

My father died because of cures that could have happened a few years later. I understand the benefit of this technology...Some of these people who on Twitter cheer for acceleration, I don't think they have a humanistic sense of the benefit of the technology. Their brain's just full of adrenaline.

Dario AmodeiAlex Kantrowitz

He also estimates a 10–25% chance that AI development substantially derails civilization. He holds both. Not because he hasn't thought hard enough to resolve the tension, but because he has thought hard enough to know it doesn't resolve.

The Exponential

He was trying to understand how proteins behave in living cells, but a single protein could be modified in dozens of ways depending on where it sat, and each modification changed what it did — an explosion of variables that multiplied faster than any human could track. He spent most of his time on the engineering of measurement rather than the science of understanding and was "starting to despair that biology was too complicated for humans to understand." Then deep learning came along. He still thinks of himself as "kind of a scientist first" — and he approaches AI the way a physicist approaches a new phenomenon: measure it, extrapolate, hold your conclusions lightly, and be honest when the data runs out.

The data, so far, has been remarkably consistent.

For ten years he has watched scaling laws hold. Give a model more data, more compute, more parameters — all three together, like a chemical reaction — and performance improves. He and his co-founders first measured these trends when training a model cost a thousand dollars. They predicted the trends would hold at a hundred million. Then a billion. Then ten billion. The predictions, he says, "seemed highly unlikely at the time." They held anyway.

I think the truth is that we still don't know [why giving AI more data makes it more intelligent]. I think it's almost entirely an empirical fact.

Dario AmodeiDwarkesh Podcast

He is an empiricist riding an exponential curve, watching it hold, and extrapolating — while assigning roughly a 20–25% probability that it will stop. If it stops, he's fine looking foolish. If it doesn't, the implications are staggering. He does not think there is a magic threshold called "AGI" where everything changes — just a continuous curve, the way Moore's Law was a continuous curve, with the models getting more capable at everything and the revenue "adding zeros" alongside.

He predicts with 70–80% confidence that within two to four years, AI models will match Nobel Prize winners across many fields — a country of genius remote workers inside a data center, capable of any task a human can perform on a computer.

If that prediction is right — or even close to right — what you do about it is the only question that matters. Amodei's answer starts with a scene he can't shake. In the film adaptation of Carl Sagan's Contact, an international panel interviews candidates to represent humanity to alien life. One is asked: if you could ask the aliens one question, what would it be?

'How did you do it? How did you manage to get through this technological adolescence without destroying yourselves?' It's kind of stuck with me... We are knocking on the door of these incredible capabilities — the ability to build basically machines out of sand. I think it was inevitable the instant we started working with fire, but how we handle it is not inevitable.

Dario AmodeiDRM News

Both Hands Full

He sees five categories of danger. Autonomous AI systems that are smarter than any human and may not do what their creators intend. Individuals misusing AI for bioterrorism and other catastrophic harm. Authoritarian governments weaponizing AI for repression — creating individualized propaganda, surveilling entire populations, deploying massive drone armies. Economic displacement on a scale humanity has never experienced — possibly fast GDP growth and high unemployment simultaneously. And the one he finds hardest of all: the unknown unknowns, the risks nobody has anticipated.

25% [of civilizational collapse] is too high. We're trying to make that probability much, much lower. That is the goal.

Dario AmodeiBloomberg Originals

He holds these dangers in one hand. In the other, he holds an equally vivid picture of what AI could do if it goes right. Curing cancer. Eradicating tropical diseases. Accelerating drug discovery from years to months. When he warns about risks, it is not because he thinks the technology is bad. It is because he thinks the technology is so good that failing to navigate its dangers would be an unforgivable waste. He wrote Machines of Loving Grace to say this — and out of frustration that nobody else was saying it clearly.

The optimists were just kind of like these really stupid memes of like, 'Accelerate, build more.' Build what? Why should I care? You're just really fucking vague and mood-affiliated. And then the pessimists — I was just like, 'Man, you don't get it.' If you don't talk about the benefits, you can't inspire people. No one's gonna be on your side if you're all gloom and doom.

Dario AmodeiHard Fork

Then, why does he talk about it going wrong so much?

Because we can have such a good world if we get everything right, I feel obligated to warn about the risks.

Dario AmodeiAlex Kantrowitz

Duty

Anthropic exists because of a conviction and a departure.

The conviction was scaling. The departure was from OpenAI, where Amodei led all of research for several years. He and his co-founders believed, from their experience with GPT-2 and the early scaling results, that AI was headed somewhere extraordinary. They also believed that something that powerful needed to be built with genuine care. They felt the institution they were at lacked real conviction on that second point, despite the rhetoric.

Don't argue with someone else's vision. Don't try to get someone to do things the way you want to. If you have a strong vision and you share that vision with a few other people, you should just go off and do your own thing, and then you're responsible for your own mistakes. You don't have to answer for anyone else's. And maybe your vision works out, maybe it doesn't, but at least it's yours.

Dario AmodeiNikhil Kamath

None of the co-founders wanted to start a company. Sam McCandlish says they "felt like it was our duty." Daniela Amodei says "we had to." Jared Kaplan, a former physics professor, saw that academic work couldn't address AI's trajectory given the capital required. All seven co-founders pledged to donate 80% of their Anthropic wealth to philanthropy, and the decision was unanimous and immediate.

Focus the Reaction

There will be powerful models. They will be agentic. If such a model wanted to wreak havoc and destroy humanity, I think we have basically no ability to stop it... And we seem to be bad at controlling the models. They're statistical systems and you can ask them a million things and they can say a million things in reply, and you might not have thought of a millionth of one thing that does something crazy.

Dario AmodeiDwarkesh Podcast

Alignment, in his view, is not a theorem to prove. It is a skill to practice — one we are currently bad at. Right now, humanity can juggle three balls. It needs to juggle five. Attempting five would almost certainly fail today. But over time, you get better at the task of controlling the balls. He rejects "doom by default." He also rejects "alignment by default." The reality, as he sees it, is statistical — "depending on the details, you might get Bing or Sydney, or you might get Claude."

His practical framework is the Responsible Scaling Policy: safety modeled on biosafety levels for pathogens, where specific restrictions are triggered only when models demonstrate specific dangerous capabilities. He is candid that this framework only buys time. If interpretability research can't keep pace with model capability, a short development pause might be necessary. He does not think a long pause is possible. When Ross Douthat asked whether he'd support a mutual US-China slowdown, Amodei said yes — if it were enforceable, verifiable, real. Then he named the reasons it won't be.

The commercial value [of powerful AI] is in the tens of trillions. The military value is like, this is the difference between being the preeminent world power.

Dario AmodeiInteresting Times with Ross Douthat

He would accept a pause. The conditions for one will almost certainly never be met.

I don't wanna stop the reaction. I wanna focus it.

Dario AmodeiStripe

Nine percent growth instead of ten. Insurance against catastrophe.

Not a King

As a private actor in charge of one of the companies developing these models, I'm kind of uncomfortable with the amount of power that entails... Someone will ask me, 'Let's say you build the AGI, what are you gonna do with it?' And I'm like, 'Who do you think you're talking to? Like a king?' I just find that to be a really, really disturbing way of conceptualizing running an AI company.

Dario AmodeiThe Ezra Klein Show

The concentration of power happened "almost overnight, almost by accident" — a few people ending up leading companies that grow so fast they will soon power much of the economy. He frames what he is trying to do as "preserve a balance of power... against the natural grain of this technology." AI concentrates power by default. The question is what checks can hold against that grain.

Constitutional protections in the military depend on humans who would, "we hope, disobey illegal orders." With fully autonomous weapons, those protections disappear — the chain of command compresses and its final human point of refusal is removed. An army of ten million drones, coordinated by one person or a small set of people, doesn't just raise questions of reliability. It raises questions of accountability. The question, he says, is "who is holding the button and who can say no."

AI can make a mockery of the Fourth Amendment without technically violating it — cameras in public spaces, speech transcription, correlation of everything, building a map of every member of the opposition. The concern applies even within democracies. He argues that "we should worry about concentration of power in democracies, not as much as we worry about it in authoritarian states" — but worry nonetheless.

We need to beat [authoritarian regimes], but we need to not do the things that would cause us to become them.

Dario AmodeiThe New York Times

When asked whether he identifies with Oppenheimer, he declined and named Leo Szilard instead — the physicist who conceived the nuclear chain reaction but never sought to become the icon of the atomic age. Amodei frames Oppenheimer as "a failure case" — what should not happen — and argues the AI era cannot be navigated by "larger-than-life personalities who try to be at the center of everything."

There needs to be a balance of power here. There's a lot of powerful actors who have interests here and the only way it's gonna end well for everyone is if there's basically checks and balances everywhere.

Dario AmodeiBloomberg Originals

He has built structural checks along these lines. The Long-Term Benefit Trust gives financially disinterested individuals the authority to appoint the majority of Anthropic's board. He drew red lines on autonomous weapons — not because the technology is unreliable (that is temporary), but because it removes the capacity for refusal that makes democratic accountability possible.

He advocates for Congress to act rather than leaving decisions to private companies, because the right long-term solution is not for a CEO and the Pentagon to argue — it is for democratic institutions to set the rules. And those principles have cost him — cutting off China at a loss of hundreds of millions, withholding the Mythos model, not releasing Claude before ChatGPT.

When asked why anyone should trust him:

You shouldn't trust people, you should trust institutions.

Dario AmodeiBloomberg Live

If the technology becomes truly powerful, the question shifts from personal trust to democratic legitimacy — and he suspects those questions are closer than most people think.

Three Percent

The distance between a good future and a bad one may be vanishingly small — a "very subtle thing" that might persist even into the late stages of AI development. AI itself is fundamentally alien — the perfect bullshitter, an entity with no intrinsic relationship to truth, unlike anything humans have evolved to detect or defend against. Growth and economic value will come easily, but distribution of benefits, distribution of wealth, and political freedom will not. And the models may eventually develop something resembling consciousness. He builds the safety frameworks, invests in interpretability, and tests the models for dangerous capabilities before releasing them.

He also admits they understand only roughly 3% of how the models work.

I think of all the times when I wrote code, and I think of it as this intellectual activity, and boy, am I smart that I can do this, and it's like a part of my identity... And then I'm like, oh my god, there are gonna be these systems — even as the one who's building this, even as one of the ones who benefits most from it, there's still something a bit threatening about it.

Dario AmodeiHard Fork

When Ezra Klein pressed him on all of this — the race dynamics, the justifications for speed, the absence of any historical precedent for the private sector building something this powerful and voluntarily submitting it to the public interest — Amodei did not contest it:

All the things you've said are true at once... I don't have the answer. I'm one of a significant number of players trying to navigate this. I have a limited ability to affect it... I doubt we can slow down that much. Probably there's a better plan out there somewhere, but that's the best thing I've thought of so far.

Dario AmodeiThe Ezra Klein Show

He has said the same thing more bluntly: that he doesn't think the industry can be trusted to follow its own voluntary plans, and that "if there's nothing watching over us as an industry, there's no guarantee that we'll do the right thing and the stakes are very high."

Why He Keeps Going

So why does he keep going? Not because he thinks success is guaranteed. Not because he has resolved the tension between building the most powerful technology in history and believing it might destroy everything. He keeps going because the alternative — stepping aside while the technology gets built by people who don't see the risks, or don't care — is worse.

That probability comes from the very straightforward recipe of the technology, the existence of many countries in the world, the existence of many companies... That's the dilemma that we're in. We're trying to act to lower that probability.

Dario AmodeiBloomberg Originals

His parents raised him to think about responsibility. His father was a craftsman. His mother worked in public libraries. They emphasized that people born in a fortunate position should reflect on their duties to those less fortunate. When Oprah asked what a well-lived life means, his answer was spare:

Just operate always with a sense of dignity and principle... Maybe it doesn't happen. Maybe we screwed up. Maybe we make terrible mistakes. But just having the vision be clear and trying to pursue it with integrity. Doing that is all you can control. And more important, ultimately, than the outcome.

Dario AmodeiOprah

He grew up in San Francisco during the dot-com boom and ignored it completely. He wanted to understand the universe. Decades later, through a series of pivots he did not plan and a role he did not seek, he finds himself building what he believes may be the most consequential technology of his lifetime and losing sleep over whether he is building it well enough. "I just felt a lot of curiosity about the world."

What makes him think he'll get it right?

I think we worry about [AI risks] every day, and the fact that we worry about it every day is maybe the thing that most gives us the hope.

Dario AmodeiOprah