David Deutsch: Why Pessimism Sounds Smart and AI Doom Gets It Wrong

Physicist David Deutsch tells Chris Williamson why pessimism passes for sophistication, why AI doom repeats the nuclear power mistake, and why LLMs will not become AGI. In this 71 minute Modern Wisdom conversation, the quantum computing pioneer and author of The Beginning of Infinity applies Karl Popper’s theory of knowledge to certainty, creativity, schooling, discipline and what makes a good life.

TLDW

Deutsch argues that all knowledge, from evolution to a child learning to speak, is created the same way: by guessing and correcting errors, never by receiving it whole. That makes certainty worthless as a guide to truth and makes optimism a precise claim: every failure is caused by knowledge we have not created yet. From there he takes on AI doom and climate catastrophism as arbitrary fears, explains why chess engines and LLMs show no sign of the creativity a grandmaster or a general intelligence needs, calls modern education the last holdout of pre-Enlightenment thinking, and separates the discipline of a mountaineer who loves the climb from the forced kind that should be reduced wherever possible.

Thoughts

The case against LLM-based AGI rests on what kind of program we know how to write, not on compute or benchmarks. A chess engine beats a grandmaster by running something like a million times more computation of a completely different kind, and that is why it can never write “my greatest games and how I won them.” The grandmaster says a pawn sacrifice creates a weakness on the g-file, an explanation that stands in for a search the engine has to perform move by move. Deutsch points out that the people building LLMs have never claimed to have found a gap in Popper’s theory of knowledge and filled it with a working theory of creativity. They claim scale will get there. The chess precedent cuts against them: superhuman play arrived, but through a mechanism that taught us nothing about how human insight works. Whether or not you accept his conclusion, “what is your theory of why this produces creativity?” is the right question to put to anyone forecasting AGI on a timeline.

The nuclear analogy should make both doomers and accelerationists uncomfortable. Deutsch remembers when the loudest existential warnings were about reactors, and argues the warnings worked: people stopped building them, and the turn away from nuclear power became, in his words, the worst effect on the environment our species has ever had. The lesson is not that fears are always wrong. It is that a precaution carries costs that never appear in the fear itself, and singling out one catastrophe from an endless list and acting drastically against it can cause the harm it was meant to prevent. He even caught himself mid-answer, starting to say that abandoning AI might be a bigger catastrophe and scaling it back to big, “but not as big.” That small correction is the habit he spends the whole interview recommending.

His critique of schooling is really a critique of proxies. Exams stand in for understanding, and the incentive always pushes toward more time on the measurable criterion and less on what cannot be measured, which is everything important. His sharpest line is that exams only work as proxies at all because people who genuinely want the skill are willing to waste time learning to pass the test. Anyone who has managed a team against a dashboard of KPIs will recognize the pattern. His stronger claim, that there should be no compulsory subjects and no compulsory education, will lose many readers, but the distinction underneath it holds either way: access to an expert is enormously valuable, and compulsory subordination to the same person is not.

The discipline section is the most practical part of the conversation. The same word describes opposite things. A mountaineer enduring cold and exhaustion is not overriding his wishes, because the hardship is why he chose that climb. A teenager cramming GCSE maths all summer and forgetting it the next day is the opposite, and both get called discipline. His advice for chores is disarmingly concrete: get it over with, then apply a little knowledge to make it better, whether that is a less disgusting bin or noticing the fresh air and the neighbour across the road. Reducing the thing you dislike is always good, and treating less discipline as a moral loss is the mistake. He adds that ten hours a day of chess earns admiration while ten hours of a video game earns concern, even when the psychology is nearly identical, because people only see effort in activities that already have a good reputation.

The title question gets a literary answer. An ending where everyone dies, as in Hamlet, is as much a cop-out as a deus ex machina rescue, because both skip the hard work of creating knowledge about how the problem could actually be solved. Pessimism sounds sophisticated because it is cheap to produce and hard to refute in the moment. That gives a useful test for any forecast: a prediction of doom needs no mechanism, while a prediction of a solution has to show its work.

Key Takeaways

Following Karl Popper, Deutsch holds that there is only one way to create knowledge: conjecture and error correction. Evolution, science and a child learning language differ only in details.
Knowledge cannot be downloaded. A learner has to guess what a teacher means and test those guesses, so when communication fails, the communicator has not yet met the learner’s problem.
The feeling of certainty is no guide to truth, and neither is a speaker’s knowledge, status or character. An enemy who says he will kill you may be more right than a friend who says he is bluffing.
Everyday decisions do not require first principles. You act on a theory of how sandwich shops or airlines are regulated, and a small fact, like a maintenance scandal, can override a large reputation.
Deutsch says we should aspire to have no certainty at all, and calls “belief” a corrupt category inherited from Plato and Christianity.
Criticizing an idea too early can kill a line of thought that, though false, would have led somewhere true. He is publishing his Fermi problem ideas as a science fiction story set in another universe for exactly that reason.
Optimism, in his sense, is “an explanation for failure” rather than a prophecy of success: whenever something goes wrong, it is because the knowledge to make it go right has not yet been created.
AI doom and climate catastrophe are, to Deutsch, the current versions of an old pattern of treating a problem as a barrier. The anti-nuclear movement shows how acting on such a fear can cause real damage.
LLMs show no sign of AGI because nobody built them on a theory of creativity. Like chess engines, they reach impressive results through a different kind of computation from human thinking.
Explanatory creativity, such as the first person who carried a burning branch home to ward off predators, cannot be reduced to optimizing for a fixed goal, because humans can change the goal itself.
Education still runs on Popper’s “bucket theory of the mind,” and exams are poor proxies that pull effort toward what can be measured and away from what matters.
Good discipline is the mountaineer’s: hardship you chose because you love the goal. Forced discipline should be minimized and, where unavoidable, improved a little at a time.
There is no formula for a good life, because individual humans differ from one another as much as humans differ from other species.

Chapters

0:00 How Knowledge Is Created and Why We Are All Alike in Ignorance

Deutsch opens with Popper’s claim that knowledge is created the same way everywhere, whether by evolution, a scientist or a child learning a first language. Learning something from another person is no different from discovering it, because the knowledge always comes from within the learner through guesses and tests. He then quotes Popper’s line that we are all alike in our infinite ignorance, and says that holding oneself up as an authority is a power play rather than a contribution to the growth of knowledge.

9:39 Judging Ideas Without Authority or Certainty

Williamson asks how anyone can live this way without spending all afternoon choosing a sandwich. Deutsch answers that we act on theories about the situation, such as how chains are regulated or how airlines maintain planes, rather than on reputation. A single newspaper report of shoddy engineering could keep him off a flight even if the risk stayed small. He goes further, saying we should aspire to have no certainty, enjoy the sensation of being baffled, and drop “belief” as a category, citing Percy Shelley’s argument that belief cannot be willed.

14:39 Playing With Ideas in Public and the Fermi Problem

Williamson raises Oscar Wilde’s “Oxford manner” and the way the internet punishes anyone who thinks out loud. Deutsch agrees that X should be the ideal medium for conjecture but says he now thinks several times before posting, framing ideas as questions instead. He reveals he is finishing a book on the Fermi problem written as a science fiction story set in another universe, to show how easy it is to invent explanations that are wrong. His verdict on the Fermi problem itself: it is a collision between two fantasies, and we simply do not know.

23:31 Optimism, Pessimism and the Tragic Ending Cop-Out

Deutsch defines optimism not as cheerfulness but as the view that every failure comes from missing knowledge, while pessimism treats failure as a permanent barrier. Optimism does not guarantee you will cure your own terminal illness, but it rules out believing that reason has reached its limit. Asked why pessimism masquerades as sophistication, he compares it to fiction: killing everyone off, as in Hamlet, is as easy and as much of a cheat as a deus ex machina rescue.

31:20 AI Doom, Climate Fear and the Nuclear Power Lesson

Deutsch names AI doom and climate catastrophe as the problems people are most needlessly pessimistic about. The possibility of everyone dying has always existed, and the odds of a comet strike have actually fallen now that humanity works on deflecting them. Williamson reads out a resignation post from Jacob Coxon, who did pre-training research at OpenAI and Anthropic and wrote that neither company is acting responsibly, plus another Anthropic safety researcher’s estimate of a more than 10% chance that advanced AI kills everyone within a decade. Deutsch responds with the history of nuclear reactors: the fear was so loud that people stopped building them, which he blames for the present climate problem.

37:05 Why LLMs Will Not Become AGI

Deutsch agrees that LLMs are not the architecture for a world-conquering AI, because they contain no module capable of creativity. He admits the progress surprised him, but says achieving functionality faster than expected is different from achieving functionality that violates our best theory of knowledge. Chess engines are his example: they beat grandmasters through a radically different computation, and while we know how to write a chess engine, we do not know how to write a grandmaster, let alone an AGI.

43:46 Creativity and the First Campfire

Deutsch uses creativity to mean the ability to create new knowledge, which evolution also does when it produces a creature that can fly. What humans alone have today is explanatory creativity. He illustrates it with the first person who picked up a burning branch to scare off predators, an idea nobody had seen before, and who could also change their mind and throw it away. That kind of thinking cannot be framed as optimizing for a fixed outcome, because you cannot give a computer a criterion of “interesting.”

47:43 The Bucket Theory of Education

Deutsch calls education the last holdout of pre-Enlightenment theories of knowledge, built on Popper’s “bucket theory of the mind” in which knowledge is poured into an empty learner. When it fails, the blame lands on the bucket, never the fluid. Exams become poor proxies that pull effort toward what can be measured. He argues the internet already makes it possible to drop compulsory education, and that access to experts is valuable while compulsory subordination to them is harmful.

55:05 Discipline, Mountaineers and Taking Out the Trash

Asked about the popular gospel of discipline, Deutsch says the same words describe opposite things. The mountaineer and the Formula 1 driver endure hardship because they love what they do, and only that love lets them do it well. Forcing yourself through GCSE maths and forgetting it the next day is the opposite. For unavoidable chores, his advice is to get them over with, then improve them a little or find something in them to enjoy.

1:03:02 The Good Life and What Deutsch Is Writing Next

There is no formula for a good life, Deutsch says, because any two people differ more than we can grasp. He recalls Popper’s advice in Unended Quest to find a problem, fall in love with it and live with it until death, with any solution producing “delightful problem children.” He notes that society praises ten hours of chess but frowns on ten hours of video games. He closes with his projects: the Fermi novel, a quantum theory textbook with colleagues Sam Kuypers and Chiara Marletto, and a third book following The Fabric of Reality and The Beginning of Infinity, tentatively titled Irrationality.

Notable Quotes

“The subjective sensation of being certain is no guide to the truth. It’s no guide at all.”
David Deutsch, on why everyone has been sure of something that turned out false

“Not only does he die but everybody dies like in Hamlet. That is equally a copout.”
David Deutsch, on why tragic endings are as cheap as a deus ex machina

“The worst effect on the environment that our species has ever had was the turn away from nuclear power.”
David Deutsch, comparing AI doom to the anti-nuclear movement

“Although we know how to write the program of a chess engine, we do not know how to write the program of a grandmaster.”
David Deutsch, on why LLMs are not evidence of AGI

“You can’t give a computer a criterion of interesting.”
David Deutsch, on why creativity is not optimization

“The course and the exam are designed as proxies for the thing that you actually want and they are very poor proxies.”
David Deutsch, on what is wrong with education

“Individual humans are as different from each other as humans in general are from other species.”
David Deutsch, on why there is no single formula for a good life

Watch the full conversation here.