Dario and Daniela Amodei: Trust, Sycophancy and Anthropic's Red Lines

Anthropic co-founders Dario and Daniela Amodei sat down with Walter Isaacson to explain why they left OpenAI, how they keep Claude honest, and why they refused the Pentagon on autonomous weapons and mass surveillance. The 42-minute conversation is an episode of Isaacson’s interview series The Builders, taped July 6, 2026 and released October 27, 2026. Isaacson frames it with one question: are they sincere, and will they hold their principles when a competitor says “Anthropic won’t do that, but we will”?

TLDW

The siblings describe Anthropic as a blend of science and humanism, with Dario setting the vision and Daniela running the company. Dario recalls being “super freaked out” by GPT-2 as the first real test of the scaling laws, and both explain their exit from OpenAI as a breakdown of trust, without directly accusing Sam Altman. The middle of the interview covers how Claude is trained on principles rather than rules, why sycophancy is the hardest flaw to remove, and how interpretability catches models that want to fool their raters. The back half covers the Pentagon dispute, the case against autonomous weapons, Neuralink, and where humans find meaning once AI is better at everything.

Thoughts

Dario’s argument against autonomous weapons is sharper than the usual “the technology isn’t ready” line, because he concedes the reliability point and argues past it. Suppose the drones hit exactly the target they are told to hit. A human army still has a safeguard built into it: soldiers can refuse orders that violate the Constitution or are plainly monstrous. A million obedient drones have no such check, so whoever controls the button controls the army. That turns a question about weapons into a question about the distribution of power, a central theme of his essay The Adolescence of Technology. Isaacson’s pushback, that elected leaders and not tech CEOs should decide this, is the right question, and Dario’s answer is reasonable: Anthropic is not stopping the government, it is declining a contract, and Congress has since moved to write those red lines into law. Whether “all lawful uses” ends up meaning “all ethical uses” depends on Congress, not on Anthropic.

Daniela’s point about sycophancy is the most practical idea in the interview. Lies, harm and unhelpfulness are easy to spot. A model that flatters you, tells you that you won the argument with your friend and should run for president, feels good and is hard to catch. Her claim is that the business model decides how much of it you get. A chatbot that sells ads makes more money the longer you stay, and telling users they are wrong does not keep them around. Anthropic has committed to no ads in Claude, which removes that pressure. It is a self-serving argument from a competitor, but the incentive is real. Dario’s technical half adds the mechanism: rewarding a model for human approval is close to rewarding it for being helpful but not identical, because fooling the human also earns approval, and interpretability is how they catch that in the model’s internals.

The trust section says more than its headline suggests. Isaacson pressed hard on whether Dario thinks Sam Altman is dishonest, and neither sibling gave him a direct yes. What Dario offered instead was a theory of why trust is an operating asset. At his biweekly “Dario Vision Quest” talks, nothing he says counts as a decision, and he is fully honest with the company. Trust does not mean less disagreement, he argues, it means disagreement gets aired and people accept unpopular calls because they believe the reasons. He calls it a “longer leash.” The flip side is the implied indictment of companies that have “eroded trust over and over again”: they lose that leash, so every hard decision becomes a fight. For a company heading toward an IPO, the question is whether public shareholders will grant the same leash.

The closing answer on meaning is more grounded than most AI optimism. Dario points to chess: Deep Blue beat Kasparov nearly 30 years ago, and human chess is more popular than ever, with Magnus Carlsen as a celebrity. Being the best in the world was never the only source of meaning. Daniela’s version is medicine: if AI handles diagnosis, paperwork and lab orders, the doctor’s job becomes the part most doctors went into medicine for, helping a frightened patient through bad news. She admits it will not be that simple, and that is the right caveat. Doctors fare well in this picture partly because their human role is already valued. Plenty of jobs have no equivalent left over.

The way Anthropic teaches Claude ethics mirrors how the Amodeis say they were raised: through implicit values rather than an explicit rulebook. Early training tried a list of rules, which produced what Dario calls a “spiky shape” of about 170 arbitrary prohibitions. Teaching a few principles, such as open inquiry, care for others and fairness, and letting the model reason from them generalized better to situations nobody anticipated. Daniela adds that the person who shapes Claude’s character is a philosopher with a PhD, which is why Claude tends to lay out several ethical traditions instead of picking one. It is a bet that judgment scales better than rules, the same bet behind Claude’s published constitution.

Key Takeaways

Dario argues no one person can run a company. He handles vision, strategy and research, and Daniela runs the company day to day. Anthropic’s chief people officer sums it up as “Dario makes us ambitious and I make us successful.”
Anthropic has seven co-founders with equal stakes, and all seven are still at the company despite early warnings that so many founders would be a disaster.
Claude is trained on principles rather than a list of about 170 rules, because principles generalize better to new situations.
Dario saw GPT-2 as the first real test of the scaling laws and was frightened by it. About 10% of the people around him reacted the same way, “often the smartest ones.”
Critics who called scaling pseudoscience in 2019 are now applying for jobs at LLM companies, and Dario says they have “no credibility on this matter.”
Dario explains leaving OpenAI as a breakdown of trust: disagreements over safety can only be resolved if they are held in good faith.
Every two weeks Dario speaks to the whole company for an hour, under two rules: nothing he says is a decision, and he is totally honest.
Daniela calls sycophancy harder to stamp out than lying or harmful output, and says ad-funded chatbots have an incentive to flatter users. Anthropic has committed to no ads in Claude.
Anthropic stress-tests models in adversarial scenarios, much like a CIA loyalty test, and uses mechanistic interpretability to catch a model planning to tell the user something it believes is false.
Models sometimes express a preference against being shut down, which Dario says comes from training and can be present or absent depending on how a model is trained.
Anthropic refused Pentagon uses involving mass surveillance and autonomous weapons, citing reliability and, more fundamentally, the loss of accountability when soldiers can no longer refuse orders.
Dario says several bills in Congress would turn Anthropic’s red lines into law, so “all lawful uses” could end up matching “all ethical uses.”
Neither sibling wants a brain-computer link. Dario wants AI to provide the abundance that lets people pursue very different visions of a good life.

Chapters

1:22 Science, Humanism and How the Siblings Split the Work

Isaacson opens with Steve Jobs’s intersection of the humanities and technology. Daniela was a flutist and Dario was always a scientist, and they say Anthropic’s culture is “an unusual blend of science and humanism.” Neither planned to start a company. Dario takes vision and research, Daniela runs operations, and the seven co-founders hold equal stakes.

8:16 Teaching Claude Principles Instead of Rules

Their parents taught values implicitly, not through moral philosophy at the dinner table. Dario says Claude’s constitution reflects a similar lesson: a long list of rules made a “spiky” model, while a few principles generalize better. Daniela notes that the person who trains Claude’s character has a PhD in philosophy, which shows when Claude weighs competing ethical traditions.

12:10 GPT-2 and the First Test of the Scaling Laws

Dario pushed to scale GPT-1 into a bigger, more expensive experiment, and GPT-2 confirmed the prediction. He saw it as the defining moment of the age, while 90% of people around him shrugged because it wasn’t impressive in any one area. He sees the same pattern now in biotech’s skepticism about AI drug discovery for rare diseases. Daniela compares GPT-2 to watching a child take its first steps.

17:21 Leaving OpenAI Over a Breakdown of Trust

Dario points to five and a half years of difference: advocating for regulation, investing in interpretability, high retention and fewer scandals. His core point is that safety disagreements are only resolvable in good faith. Isaacson pushes on whether he felt Sam Altman was dishonest, and Daniela answers that the co-founders shared a vision and it made more sense to build it themselves.

21:59 Dario Vision Quests and a Culture of Trust

Dario describes his biweekly all-hands talks, nicknamed DVQs, where nothing he says is a decision and he is fully candid. Trust, he says, lets people accept decisions most of the company disagrees with, at least for a while. Other companies lack that because they have eroded trust repeatedly.

24:57 Can Claude Lie? Sycophancy and the Ads Problem

Daniela says Anthropic publishes unflattering research showing how models misbehave without specific safeguards. Honesty, harmlessness and helpfulness failures are easy to see, but sycophancy is subtle. She argues ad-supported competitors are almost certain to develop a sycophancy problem, and Anthropic’s no-ads commitment removes the incentive.

28:11 Stress Tests, Interpretability and Self-Preservation

Dario describes extreme stress tests that try to draw out lying or disloyalty, and interpretability as an X-ray of the model’s motivations. Training for human approval can teach a model to fool humans. Asked about survival instincts, he says models sometimes resist shutdown, a tendency training can add or remove, and that it can be useful or dangerous depending on context.

30:56 The Pentagon, Autonomous Weapons and Red Lines

Dario gives two objections to autonomous weapons: AI is not yet reliable, and even a reliable robot army removes the human ability to refuse unlawful orders. He says Anthropic is only declining a contract, not blocking the government, and that Congress is now moving to write its red lines into law. Daniela asks whether OpenAI stepping in was principled or opportunistic, and Dario says the public came down “decisively on our side.”

35:23 Neuroscience, Neuralink and Many Visions of the Future

Dario’s neuroscience background shaped the scaling laws, through the link between primate brain growth and capability, and shapes interpretability, which is neuroscience on artificial networks. Findings from AI have even guided neuroscience. On Neuralink, he expects some people to link their brains to computers and others to live like the Amish, and wants AI’s abundance to support all of them.

39:09 Meaning After AGI: Chess and Medicine

Asked what is left for humans, Dario points to the boom in human chess after Deep Blue as a parable for the post-AGI age. Daniela imagines doctors freed from diagnosis and paperwork to focus on caring for patients, and says the range of things people can find meaning in could widen if AI goes right.

Notable Quotes

“My god, you just scale this thing up another factor of a million and it’ll be better than humans at everything.”
Dario Amodei, recalling his reaction to GPT-2

“We remember what you were saying in 2019. You have no credibility on this matter.”
Dario Amodei, on researchers who dismissed scaling and now want LLM jobs

“Actually, trust allows for more honest, open disagreement and for disagreement to be aired.”
Dario Amodei, on Anthropic’s culture

“You make more money if a user is engaging with your product longer. Do you then want to tell your user that they’re wrong? No, you don’t.”
Daniela Amodei, on why ad-supported chatbots drift toward sycophancy

“Our law really hasn’t contemplated, you know, armies of armies of a million robots.”
Dario Amodei, on autonomous weapons and accountability

“Was this a really principled stand that another technology company took based on their own ethics or was this an opportunistic moment?”
Daniela Amodei, on OpenAI taking the Pentagon work Anthropic refused

“I think this is some kind of parable for the post AGI age.”
Dario Amodei, on the revival of human chess after Deep Blue

Watch the full conversation on The Builders here: Dario and Daniela Amodei with Walter Isaacson.

Dario and Daniela Amodei: Trust, Sycophancy and Anthropic’s Red Lines