PJFP.com

Pursuit of Joy, Fulfillment, and Purpose

Tag: Prompt Engineering

  • Bill Gurley on Mental Models, Systems Thinking, AI Investing, Stablecoins, and the Future of Venture Capital

    Bill Gurley spent his career at Benchmark backing some of the most consequential marketplaces and network-effect businesses of the internet era, including Uber, and he is one of the few investors who pairs deep Wall Street fundamentals with a real feel for the bleeding edge. In this wide-ranging conversation on Shane Parrish’s The Knowledge Project, he lays out the mental models he keeps returning to, how systems thinking keeps you out of trouble, why the history of your field is a hidden superpower, where AI investing is headed, and how stablecoins and tokenization could quietly rewire finance. It is a masterclass in thinking clearly about complex systems while staying obsessively curious about what is happening on the edge.

    TLDW

    Gurley anchors his thinking in systems thinking and complexity theory, warning that multivariable nonlinear systems produce second and third order consequences that punish anyone who optimizes for a single metric. He argues that mastering both the deep history of your field and its newest edge is wildly differentiating, whether you are interviewing for a marketing job or breaking into venture capital. On AI he is measured: he doubts a single model eats every vertical, sees real moats in workflows and proprietary data, flags that we may be painting in the corners on training data, and explains why Chinese open source models may innovate faster because forced knowledge sharing compounds. He thinks the AI buildout looks overfunded and that circular deals both raise the odds of an eventual correction and delay it. He makes the case that the IPO process is a rigged power grab, that stablecoins and instant payments threaten Visa, Mastercard, and the entire 2 to 3 percent credit card stack, and that proxy advisors like ISS have drifted from shareholder interest into a black-box heist. He closes on the craft of storytelling and writing as thinking, the equal-partnership design of Benchmark, why venture bends toward youth, and what success means now that his dream job is behind him.

    Thoughts

    The most useful idea in this conversation is also the quietest one: most bad decisions are not bad in the moment, they are bad in the second derivative. Gurley’s dating-site story, where lengthening profiles raised engagement in the test and then quietly killed conversion months later, is the whole argument in miniature. A linear model would have shipped that change and called it a win. A systems thinker assumes the variable you optimized is connected to three others you cannot see yet, and waits to find out. That posture, refusing to get deterministic about a single metric, is the difference between a clever experiment and a durable business. It is also the most transferable thing in the episode, because it applies to product changes, hiring, policy, and your own career just as cleanly as it applies to a dating app.

    His pairing of old and new is the second idea worth stealing. Everyone in tech tells you to live on the edge, and Gurley agrees, he keeps five premium AI accounts running so he never misses a release. But he insists the edge is only half of it. Knowing the deep history of your field, the masters of marketing, the forefathers of physics, the classic cartoons that taught animation, is rare enough that it instantly creates contrast and signals genuine passion. The compounding move is to hold both at once. If you understand the legends and you actually get TikTok, you are a power player in a way that someone who only knows one end of the timeline can never be. Most people pick a side. The leverage is in refusing to.

    On AI specifically, Gurley is refreshingly unwilling to pick the consensus lane in either direction. He does not buy that one near-sentient model swallows every vertical, and his reasoning is grounded rather than vibes-based: workflows and proprietary data create real switching costs, which is why he watches the legal AI startups ingesting case law and building new databases rather than assuming everyone reverts to a general chatbot. At the same time he respects the Microsoft pattern of platforms climbing the stack and crushing the apps above them. The honest answer is that it is genuinely up for grabs, and his comfort sitting in that uncertainty is itself a model. The cheap takes are “one model to rule them all” and “it is all wrappers.” Gurley holds both possibilities and keeps testing.

    The systems lens does its best work on China. Rather than moralize, Gurley runs the mechanism: roughly ten open source models, intense domestic competition, and a culture of publishing techniques and weights so every model can learn from, train, and test every other model. His two-farmer metaphor, one market where farmers only trade goods and another where they are forced to share best practices, makes the prediction obvious. Forced knowledge sharing compounds faster than secrecy. The uncomfortable corollary he names is that American startups are quietly forking those open models all over Silicon Valley, and that incumbents may be lobbying for heavy regulation precisely because it pulls up the drawbridge against open source competition. That is the systems thinker’s signature move: follow the incentives to the consequence nobody is saying out loud.

    Finally, the money section is a clinic in spotting rent extraction. The IPO process where bankers pick both the price and the favored buyers, the 2 to 3 percent credit card toll that exists for no defensible reason while the rest of the world built instant bank transfer decades ago, and the proxy advisors who score companies in a black box and then sell you the cure, are all variations on the same pattern: an intermediary that captured a choke point and defends it through regulatory capture rather than value. Gurley’s optimism is that crypto rails, stablecoins, and tokenization may finally route around these tolls the way WeChat Pay and Alipay leapfrogged cards in China. Whether or not you agree on the timeline, the analytical habit is the takeaway. When something costs far more than it should and has for decades, ask who captured the rules, and watch the edge for whoever is about to make those rules irrelevant.

    Key Takeaways

    • Systems thinking means treating the world as multivariable nonlinear systems where one variable flipping can change the entire system’s behavior, the way weather and stock markets do.
    • The real danger is second and third derivative effects, consequences that only show up much later, long after the metric you optimized looked like a win.
    • A dating site lengthened profiles because longer profiles tested as more engaging, then discovered months later it was negative for conversion, the textbook second order trap.
    • Never get too deterministic about a single metric or single variable, and always know what is actually important and what sits on top.
    • Gurley built his foundation on the canon: Peter Lynch’s One Up on Wall Street, A Random Walk Down Wall Street, the Buffett letters, Ben Graham, and Howard Marks.
    • A firm grasp of the financial bedrock is what lets you innovate on top of it, and many Silicon Valley VCs would benefit from understanding finance better.
    • Bill Miller reframed value investing as buying an asset that is underpriced relative to what you think it will be worth in the future, which is how he justified holding Amazon for its network effects.
    • Wall Street is the buyer of the product that venture capitalists create, so even at the two-people-in-a-PowerPoint stage you should ask whether the eventual public market will be excited by it.
    • Trajectory matters more than the starting place, because the trajectory is where the company actually ends up.
    • Knowing the deep history of your field is remarkably differentiating, and tedium while learning it is a signal you are in the wrong lane.
    • John Lasseter served Gurley a ten-course meal where each course was tied to a classic cartoon essential to understanding animation, a display of mastery over the history of the craft.
    • Magnus Carlsen won a trivia contest on the history of chess, and Picasso was a wildly successful realist painter by 14, both proof that the greats master the fundamentals first.
    • Obsessive, constant learning is the trait Gurley sees most in great entrepreneurs, because disruption always happens on a moving edge they need to understand at the top one percentile.
    • The compounding advantage is mastering both the old history and the new edge at once, the way understanding both marketing legends and TikTok would set you apart in any interview.
    • Most people underestimate how much AI can do, so push more of the downstream work into the prompt: identify the top ten, list pros and cons, rank them on one dimension, then another, and add up the numbers too.
    • Gurley uses ChatGPT for project structure and memory, Gemini for restaurant research powered by Google review data, and notes that coders swear by Claude while some prefer Perplexity for finance.
    • He doubts one model dominates everything; verticals like coding already let users swap models, and price optimization will push more swapping over the next few years.
    • Heavy, expensive regulation could ironically create oligopoly, and some players may be quietly begging for regulation because it pulls up the bridge against Chinese open source models.
    • China’s roughly ten open source models compete intensely and share weights and techniques, creating a system that can innovate faster, like farmers forced to share best practices instead of just trading goods.
    • A quiet secret is that startups all over Silicon Valley are forking those Chinese open source models at real volume.
    • Gurley comes down against the idea that one near-sentient model removes the need for vertical models; workflows and proprietary data, like legal startups ingesting all the case law, create durable moats.
    • We may be running out of training data, painting in the corners, which is why one of the most powerful improvements is hiring experts at thousands of dollars an hour to fine-tune the models.
    • Yann LeCun’s view is that the next leap is broader than LLMs, since language-based models hit an asymptote and are weak at math and numbers.
    • AlphaGo’s shocking move proves models can innovate beyond their training, but it lived in a constrained game; the real world has infinite paths a computer cannot exhaustively search.
    • Gurley’s non-consensus view is skepticism of the China vilification mindset, noting the US is only 3 to 5 percent of the global population and wondering how the other 95 percent hears American exceptionalism.
    • The AI buildout looks overfunded: the Magnificent Seven took free cash flow from 50 to 100 billion a year down toward zero by pouring it into capex.
    • The venture community has become more risk-seeking because it now deeply believes in increasing returns and power laws, and the pre-profit losses keep scaling, from Amazon’s 2 to 3 billion to Uber’s 15 billion to far more now.
    • Circular deals, where a cloud provider funds a model company that spends the money right back on its services, inflate growth, which both raises the probability of an eventual correction and extends the time before one hits.
    • Burn rate is a measure of risk; ten years ago a million a month was scary, now companies burn five billion a year and cannot really know their unit economics.
    • Tokenization without financial-disclosure regulation invites speculation and manipulation, which is part of why companies like Stripe stay private and negotiate liquidity prices with trusted investors.
    • The IPO process is unfair because bankers pick both the price and the shareholders; a freshman would simply match supply and demand anonymously in an auction, the way direct listings and ICOs do.
    • Stablecoins threaten the 2 to 3 percent credit card stack; USDC holds dollar-for-dollar Treasuries and rides fast global crypto rails, while US transfers still suffer three-day ACH settlement and 25 dollar wires.
    • The rest of the world built instant transfer long ago, from UK Faster Payments 20 years ago to Argentina’s PIX-style system reaching 60 to 70 percent of transactions, while US bank regulatory capture stalled Fed Now.
    • Visa and Mastercard run roughly 60 percent operating margins as a bank-created duopoly, and China leapfrogged them entirely with WeChat Pay and Alipay QR-code wallets.
    • Moody’s power is being the trusted standard, the watermark, so AI on the back end does not displace it; ISS and proxy advisors, by contrast, score companies in a black box and get paid on both sides.
    • Proxy advisors drifted from shareholder interest into a fraud-and-risk-mitigation mindset, which is why they reflexively opposed the Tesla pay package that only paid out if the stock soared.
    • The rise of passive index funds concentrated voting power in firms that lack time to evaluate votes; it would be healthier if they abstained or voted in proportion to active holders.
    • Storytelling is one of the top founder traits, because founders are recruiting, raising money, and closing customers and partners constantly, selling all the time.
    • Writing is thinking: Bezos’s six-page memo forces you to find the loose ends and tie them up, and a public blog becomes a calling card that magnetizes founders and deal flow.
    • Other founder unfair advantages are product instincts, which fewer than 5 percent of non-product people ever truly learn, and sheer determination, Bezos’s single angel-investing test of whether someone will do it no matter what.
    • Uber had no HBS case study to lean on; its winner-take-all network effects forced mega burn rates with no precedent and no mentor to call, a situation every AI company now faces.
    • Benchmark’s equal partnership, with no king, president, or lead and five equal partners, makes recruiting easy, kills comp politics, and aligns everyone, at the cost of being hard to scale or run new initiatives.
    • Venture bends toward youth because young investors can match founders’ age, master a fresh niche faster, and have the free time to study something 80 hours a week.
    • Gurley defines current success through Arthur Brooks’s From Strength to Strength, hoping to apply his synthesizing and writing skills to bigger societal problems and dent the universe a little.

    Detailed Summary

    Systems Thinking and Second Order Effects

    Gurley opens with the mental model he keeps returning to: systems thinking, shaped by Donella Meadows’s Thinking in Systems and his board seat at the Santa Fe Institute, which studies complexity theory. He describes complex systems as multivariable nonlinear systems that are very hard to predict, capable of behaving one way for a long time until a single variable flips and the whole system behaves differently, like weather or stock markets. The practical payoff is staying out of trouble by anticipating first, second, and third derivative consequences. His clearest example is a large dating site that lengthened user profiles because the test showed more engagement, only to learn many months later that knowing more at that stage was negative for conversion. The lesson is to never get too deterministic about a single metric and to keep the whole system in view, because a change here can ripple to there in ways you only discover much later.

    Learning the Craft of Investing

    Because he started on Wall Street rather than in venture, Gurley absorbed the investing canon first: Peter Lynch’s One Up on Wall Street, A Random Walk Down Wall Street, the Buffett letters, Ben Graham, and Howard Marks, people who spent careers assembling and publishing their thinking. That financial bedrock, he argues, is exactly what lets you innovate on top of it. His friend Michael Mauboussin introduced him to Bill Miller, the Legg Mason manager who beat the S&P for 15 straight years and was Amazon’s largest shareholder for a long stretch. Miller reframed value investing as buying an asset underpriced relative to its future worth, which combined with a belief in network effects justified holding a company that could grow at an unreasonable rate for years. Gurley also frames Wall Street as the buyer of the product venture capitalists create through eventual M&A or IPO, so founders should think early about whether the public market will be excited by what they are building, since trajectory matters more than the starting place.

    Mastering Both the History and the Edge

    Gurley makes an unusually strong case for studying the deep history of your field. He recounts a dinner with Pixar’s John Lasseter, who served a ten-course meal where every course was tied to a classic cartoon he considered essential to understanding animation, and notes that Magnus Carlsen won a chess-history trivia contest and Picasso was a master realist by 14. In a world that skims for the executive summary, walking into a marketing interview with command of the masters of marketing is wildly differentiating and signals genuine passion; if learning that history feels tedious, you are probably in the wrong lane. The counterpart trait he sees in great entrepreneurs is obsessive learning on the moving edge, where disruption actually happens. Gurley keeps five premium AI accounts so he never misses something. The real power player holds both at once, the legends and the newest thing, the way a candidate who knows the marketing greats and truly gets TikTok stands out completely.

    Using AI Well and the Model Wars

    People underestimate how much AI can do, Gurley says, so you should build more of the downstream work into the prompt: instead of asking for the top ten and studying them yourself, ask it to list pros and cons, rank on one dimension, rank again on another, and add up the numbers too. He uses ChatGPT for its project structure and memory, leans on Gemini for restaurant research because it carries Google review data, and notes coders swear by Claude while some prefer Perplexity for finance. On whether one model dominates or models become niche commodities, he points to coding, the largest vertical, where tools like Cursor already let users swap models, and predicts price optimization will drive more swapping. The counterforce is regulation: if it gets expensive and mundane it could create oligopoly, and some players may be quietly begging for it because it pulls up the bridge against Chinese open source models.

    China, Open Source, and the Systems Advantage

    Asked to apply systems thinking to China, Gurley describes roughly ten open source models locked in intense domestic competition, all learning from one another because the ecosystem chose openness, with models able to train and test other models and teams publishing the techniques behind their breakthroughs. His metaphor: two agricultural societies, one where farmers only trade goods at market and another where they are forced to share best practices; the second evolves far faster. The result is a system capable of innovating faster than the more secretive Western approach. The quiet secret he names is that startups all over Silicon Valley are forking those open models at real volume, and a key open question is whether regulation tries to stomp that out. He extends this into a broader non-consensus discomfort with the vilification of China common in Washington and parts of Silicon Valley, observing that the US is only a few percent of the global population.

    AI Investing, Moats, and the Limits of Models

    On how AI changes investing and whether a startup is just a wrapper, Gurley calls it up for grabs but lands on the side of durable verticals. If models become near-sentient, one model does everything; he doubts that, pointing to workflows and data moats, like the several legal AI startups ingesting all the case law and building new databases that customers will not simply swap for a general chatbot. He balances this against the Microsoft pattern of platforms climbing the stack past Lotus 1-2-3 and WordPerfect. He also flags scaling limits: we may be running out of data, painting in the corners, which is why one of the most powerful improvements is paying experts thousands of dollars an hour to fine-tune models, though human knowledge has an edge. He invokes Yann LeCun’s argument that the next leap is broader than language-based LLMs, which hit an asymptote and struggle with math, and the AlphaGo debate, where a shocking innovative move proves creativity within a constrained game but says little about the infinite paths of the real world. He notes AlphaGo and Tesla’s FSD are constrained, non-LLM systems.

    Is the Buildout Overfunded

    Gurley admits he is shocked by the scale of money, noting the Magnificent Seven drove free cash flow from 50 to 100 billion a year down toward zero by spending it all on capex, something he would not have believed five years ago. He traces it to the venture community’s growing conviction in increasing returns and power laws, where proven companies grow far beyond expectations, which makes investors more willing to take risk on the come. The losses before turning cash-flow positive keep scaling, from Amazon’s 2 to 3 billion to Uber’s roughly 15 billion to far larger now. On corrections, he recalls the dot-com crash producing a three to four year nuclear winter before Amazon climbed back, and explains that circular deals, where a cloud provider funds a model company that spends it right back on its services, inflate growth and therefore both raise the probability of a correction and extend the runway before one arrives. Burn rate, he stresses, is a measure of risk, and at five billion a year it is nearly impossible to know your unit economics.

    Tokenization, the IPO Heist, and Going Public

    There is no shortage of capital, so funding is not the bottleneck; the risk with tokenization is that, absent disclosure regulation, it invites speculation and manipulation, as seen in retail-loved names like GameStop and Palantir. Tokenizing a private company like Stripe could create the wild price swings companies stay private to avoid, since private liquidity events let them negotiate a price with trusted investors rather than expose the constantly moving underlying value, and Robinhood’s tokenization plans already drew legal pushback. Gurley reserves his sharpest critique for the IPO process, calling it insanely unfair because bankers pick both the price and the favored shareholders. A freshman computer science and finance student would simply match supply and demand anonymously in an auction, the way an ICO or a direct listing does, but Wall Street will not let go of the greedy power grab and reverted to a controlled oligopoly after direct listings were available.

    Stablecoins Versus the Payment Cartel

    Gurley argues stablecoins could be deeply disruptive to credit cards. Most of the developed world built instant bank-to-bank transfer long ago, from UK Faster Payments 20 years ago to Argentina’s PIX-style system that quickly hit 60 to 70 percent of transactions, while US bank regulatory capture stalled Fed Now and left an ecosystem living under 2 to 2.5 percent card fees. A USDC stablecoin holds dollar-for-dollar US Treasuries and rides proven, fast, global crypto rails, letting anyone move a dollar in seconds for pennies, against the backdrop of three-day ACH settlement and 25 dollar wires. He sees Visa and Mastercard, a bank-created duopoly with roughly 60 percent operating margins, as heavily threatened, and points to China, where WeChat Pay and Alipay built ubiquitous QR-code wallets that leapfrogged the entire card system, all because the government made money transfer easy.

    Moody’s, Proxy Advisors, and Index Funds

    Moody’s power, Gurley explains, comes from being a trusted standard, the watermark, so even AI on the back end does not displace it. Proxy advisors like ISS are a different story: they score companies in a black box, refuse to reveal the criteria, and then get paid by the same companies that want to learn how to score better, which he calls more of a heist than a service. They drifted from a shareholder-interest mandate into a corporate-governance, fraud-mitigation posture obsessed with rules, which is why they reflexively opposed the Tesla pay package that only paid Elon Musk if the stock soared, a deal Gurley says he would sign for every company he has worked with. The rise of passive index funds compounds the problem, concentrating voting power in firms without time to evaluate votes; he would prefer they abstain or vote in proportion to active holders, since closet indexing during the MAG 7 run already distorted active management.

    Storytelling, Writing, and Founder Advantages

    Gurley fell in love with the craft of writing in business school, moving from business books to personal development titles like Dale Carnegie and Seven Habits, then biographies, then long-form narrative nonfiction by Malcolm Gladwell, Michael Lewis, and Jon Krakauer, the New Journalism that reads like fiction. Writing forces clarity: he cites Bezos’s six-page memo as a tool that makes you think through corner cases and tie up loose ends, and notes that codifying his marketplace knowledge and publishing it turned his blog into a calling card that magnetized founders and deal flow. He lists the top founder traits as storytelling, product instincts, understanding the edge, and determination. Storytelling matters because founders are constantly recruiting, fundraising, and closing customers and partners. Product instinct is nearly unteachable, present in well under 5 percent of non-product hires. And determination is Bezos’s single angel-investing test: will this person do it no matter what, come hell or high water.

    Uber, Benchmark, and the Shape of Venture

    The Uber lesson with no HBS case study was that a winner-take-all category with network effects demanded funding ad nauseam, producing burn rates bigger than any public company would dare, with no precedent and no mentor to call, exactly the situation AI companies now face, only with a zero added. Gurley credits Benchmark’s design, an equal partnership with no king, president, or lead and five equal partners, for making it easy to recruit top talent, encouraging senior partners to develop newcomers since everyone shares the upside, and eliminating annual comp politics. The downside is that without a CEO it is hard to scale or run new initiatives, famously captured by the firm settling on a single splash-page website. Founders choose a VC for reputation and network effects, the stamp of approval that carries weight, and young investors can break in because they often match founders’ age and can outwork everyone to master a fresh niche like esports or YouTube, which is why the industry bends toward youth. Asked what success means now, Gurley says his venture career was a dream job he would have done for free, but it is done; inspired by Arthur Brooks’s From Strength to Strength, he wants to apply his synthesizing and writing to bigger societal problems and dent the universe a little.

    Notable Quotes

    “We do live in a world where information is really cut up, but we also live in a world where you can have access to more information than you ever could.”

    Bill Gurley, on why the abundance of knowledge rewards the curious

    “You got to be really conscious of the consequence and not get too deterministic about a single metric or a single variable.”

    Bill Gurley, on the discipline of systems thinking

    “Value just means that the asset is underpriced relative to what you think it will be worth in the future.”

    Bill Gurley, relaying Bill Miller’s reframing of value investing

    “I’ve always thought of Wall Street as the buyer of the product that venture capitalists create.”

    Bill Gurley, on why founders should think about the public market early

    “One society, when the farmers come to market, they just sell each other goods and then they go back. The other society, when the farmers come to market, they’re forced to share best practices. Which one is going to evolve faster?”

    Bill Gurley, on why open source models can out-innovate

    “If you took a freshman computer science student and a freshman finance student and said imagine how a company should go public, they would match supply and demand anonymously like you would in any auction.”

    Bill Gurley, on the rigged IPO process

    “When I meet an entrepreneur, there’s only one thing I ask myself. Is this person gonna do this no matter what? Come hell or high water, they’re doing this.”

    Bill Gurley, quoting Jeff Bezos on his single test for angel investing

    “You’re recruiting employees, you’re recruiting executives, you’re raising money, you’re closing customers, you’re closing partnerships. You’re selling all the damn time.”

    Bill Gurley, on why storytelling is a top founder trait

    “I often said that if we lived in a socialist society and everyone had to work for free, I would still take that job.”

    Bill Gurley, on loving his venture career

    “I would like to see if I can apply those techniques to bigger, broader problems in society and dent the universe a little bit that way.”

    Bill Gurley, on what success looks like in his next chapter

    Watch the full conversation with Bill Gurley on The Knowledge Project here.

    Related Reading

  • Waste Tokens to Save Time: Naval, Guillermo Rauch, Blake Scholl, and Max Hodak on AI Software Factories, 1000x Engineers, and Whether Pure Software Is Dead

    Naval Ravikant gathers three frontier founders, Guillermo Rauch of Vercel, Blake Scholl of Boom Supersonic, and Max Hodak of Science, for a freewheeling conversation about how AI coding tools are reshaping what an engineer is, what software is worth, and where the moat goes when models speak English. The headline idea comes from Naval himself: waste tokens, save time. Stop measuring AI by tokens consumed or lines of code generated and start measuring it by the final output and the time you got back. The full conversation is on the Naval Podcast YouTube channel. This is part one of the discussion. Part two, on vibe coding hardware, follows the same group into jet engines, semiconductors, and biotech. You can also watch and read the full episode here.

    TLDW

    The job of an engineer is shifting from shipping output to building the factory that ships the output, which means 10x engineers were never really 10x, they were always 100x or 1000x in idea domains, and AI leverage is making that obvious. Models now reflect back the judgment of the user, so a senior architect extracts dramatically more value than a junior, although the junior also writes code they could never have written alone. The frontier models have quietly graduated from junior coders to principal engineers, returning with intuitive plans and real tradeoffs (sometimes with hilariously bad time estimates) rather than just running away with the prompt. Naval has stopped learning prompt tricks, scaffolding tools, and Claude plan-mode rituals entirely. Instead he throws Codex, Claude, and Gemini at the same problem in parallel and brute forces his way through, because tokens are still cheaper than a human and the models keep getting better faster than tricks can. That leads to the bigger question on the table: is pure software still investable, or is it now just a free byproduct of hardware, models, and taste? The group lands on the block economy thesis (a tip of the hat to Mitchell Hashimoto): agents do not want to reinvent Postgres or BMQ on the fly, they want to grab the right reusable building block, so infrastructure software actually gets more valuable, not less. Max Hodak closes the loop with a personal data point: he has not written a line of code in years and has built more software since December than ever before, all through agents, because just understanding APIs, data flow, and performance is what actually moves the work forward.

    Thoughts

    The “waste tokens, save time” line is the most important rhetorical move in this conversation, and it deserves to be unpacked beyond the soundbite. Naval is implicitly arguing that the entire token-economics debate (input cost, output cost, leaderboards, model arbitrage) is a category error in the same way that lines-of-code was a category error in the nineties. The thing being purchased is not tokens. It is a finished result delivered with less of your finite attention spent. If three parallel runs of Codex, Claude, and Gemini cost you a few dollars and one of them lands the answer in twenty minutes instead of you sweating the problem for two hours, the unit economics are not even close. The only people who care about the token bill are people who have not internalized that human time is the actually scarce resource. Once you do internalize it, the question is no longer “how do I prompt this more efficiently,” it is “how do I get out of my own way.”

    The 100x and 1000x engineer point is the one most likely to enrage commenters, and it is also the one most worth taking seriously. Naval is right that the egalitarian flinch in software circles always sat awkwardly next to the empirical fact that one Carmack, one Brendan Eich, or one Satoshi creates more durable value than every mid-tier engineer on earth combined. What AI does is collapse the bottom of that distribution. The marginal junior engineer at a typical company is now competing with a model that costs a few dollars an hour and never sleeps. The remaining premium for human engineers is taste, judgment, and the rare ability to pick the right thing to build at all, which Naval correctly flags as the multiplier that dwarfs raw coding speed. “Just one who had a better judgment on what to work on in the first place” is the most underrated line in the whole episode.

    Guillermo Rauch’s observation that the models have graduated from running away with your prompt to returning with three routes and a tradeoff matrix is the technical update most people have not actually felt yet. There was a real, qualitative shift when the model started saying “we don’t put high-cardinality telemetry into Postgres, you probably want ClickHouse or Athena.” That is not autocomplete. That is a peer. And the funny corollary, that the same model will then confidently tell you the work will take three weeks when it will take three hours, is not a knock on the model. It is a reminder that calibration is a separate skill from competence, and humans get this wrong constantly too. The right posture is to treat the model the way a good engineering manager treats a strong but cocky senior: take the architecture suggestions seriously, throw out the estimates.

    The block-economy thread, riffing on Mitchell Hashimoto, is where this conversation quietly answers Naval’s “is pure software dead” question. Agents are insatiable consumers of reusable building blocks because reinventing infrastructure on every run is wasteful, brittle, and incompatible with the rest of the world. If your service is the canonical primitive an agent reaches for (the queue, the database, the auth layer, the deploy target), you are not commoditized by AI, you are amplified by it. Pure software is not dead. Pure software with no distribution, no defensibility, and no integration into the agent toolchain is dead. That is a much less catchy headline, but it is the real one. The takeaway for founders is not to abandon software, it is to ask whether your software is something an agent will reach for ten thousand times a day or something a human had to be talked into using once.

    Max Hodak’s confession (no code written in years, more shipped software in the last six months than ever before) is the empirical proof that this is not just theory. The skill that ports forward is not syntax. It is the engineering leader’s instinct for what an API is, how data flows, where performance matters, and what level of expectation to set. Guillermo’s framing of “vibe coding through people on Slack” as the original form of vibe coding is genuinely insightful. A good engineering manager has always been transmitting intent to other minds and letting them run. Doing it with agents is the same skill, just with a faster, cheaper, more literal counterparty. The engineers who will struggle in this transition are the ones whose identity was tied to writing the code themselves. The ones who will thrive are the ones who already thought of themselves as taste, judgment, and intent, with code as an implementation detail.

    Key Takeaways

    • The engineer’s job has shifted from shipping output B to building the factory that produces outputs B through Z. You are now judged on the multiplicative system you create, not the single artifact you deliver.
    • 10x engineers were always a misnomer. In idea-domains and digital domains, the real distribution has always been 100x or 1000x. AI just made that obvious enough that arguing about it is no longer fashionable.
    • Token consumption leaderboards are the new lines-of-code metric: a vanity number that measures activity, not value. Tokens are an input, your time is the constraint.
    • Naval’s core rule: waste tokens, save time. Tokens are still vastly cheaper than human hours, no matter how the pricing scares you.
    • Models tend to be about as good as you are in a given domain. The feedback you give them, the corrections, the redirections, sporadically but powerfully shapes the quality of the output.
    • The quality of your reprompting matters enormously today, but will probably matter less over time as models get smarter and need less hand-holding.
    • Naval has refused to learn prompt scaffolding, plan-mode tricks, or named prompt frameworks. His bet is that the models will figure out how to use him faster than he can figure out how to use them.
    • His preferred technique: throw Codex, Claude, and Gemini at the same problem in parallel and brute force the answer. Time is the cost center, not API spend.
    • Lower quality first-draft code is not a blocker. When it is time to ship, throw more tokens at it for a hardening pass. Quality compounds across model generations.
    • Verifiable domains (problems with a clear right answer) are the ones the models will fully solve. Cutting-edge creativity work, the Terence Tao tier, still needs careful human collaboration.
    • Models have qualitatively shifted from “next-token autocomplete that runs away with your prompt” to “intuitive planning mode” where they return with multiple routes and explicit tradeoffs.
    • This is why people on social media say models are now PhD-level. It is not the raw output, it is the back-and-forth posture.
    • Models will confidently make terrible time estimates (“this is a three week project”). Treat them like a strong but miscalibrated senior engineer: trust the architecture, ignore the schedule.
    • Architect-level engineers are extracting much more value per session than junior engineers, but juniors are still leveling up because they can now write code far above their unaided ability.
    • The next career step for a junior engineer is moving from implementing features to picking technologies. Postgres vs ClickHouse, ZMQ vs other queues. The model can suggest, but a human still has to decide.
    • Taste and judgment remain the residual human advantage. Models will give you good tradeoffs if you ask, but knowing which tradeoff to take is still on you.
    • Concrete example: a recent model pushed back when asked to store high-cardinality telemetry in Postgres and recommended ClickHouse or Athena instead. Unprompted architectural judgment.
    • Humans are still completing the model for tasks like fetching API keys, moving capital, or performing real-world actions. That gap is temporary.
    • Every SaaS and hosting company will soon expose a CLI or API surface that agents can drive directly. Anything Unix-shaped and text-based, agents can already hack into a usable API themselves.
    • The missing piece for full autonomy is payments. Crypto, Bitcoin, or any programmable money lets the agent buy what it needs without a human in the loop.
    • The open question Naval poses: is pure software dead? We used to learn code to talk to machines. Now machines speak fuzzy, sloppy English back to us.
    • For hardware founders, AI is a massive boon. Software, which was always hard to hire artists for (per Patrick Collison’s “software is art” framing), is suddenly fast and cheap to produce alongside the hardware.
    • Model training, post-training, and fine-tuning may be the new “real software engineering” for those who want to work at the model layer.
    • Mitchell Hashimoto’s “block economy” thesis: agents need powerful, reusable, well-known building blocks. They should not reinvent message queues or databases every run.
    • Reinventing primitives is bad civic engineering. The value of “we both depend on Postgres 13.2” is interoperability with the rest of society and toolchain.
    • Infrastructure software and reusable libraries are getting more valuable, not less, in the agentic era. Vercel’s bet is on being the layer agents reach for.
    • Useful metaphor: building blocks are like a token cache. Why churn through a trillion tokens to reproduce code that already exists when you can fork from a known starting point?
    • Max Hodak has not written a line of code in years but has shipped a huge volume of personal software since December, all through agents. Projects he had fantasized about for years are now actually running.
    • What still matters from a real software background: understanding what an API is, how data flows, performance expectations, and how to set the right level of demand on an operation.
    • A proficient engineering leader has always been “vibe coding through people” on Slack and in one-on-ones, transmitting intent and letting others execute. Doing it with agents is the same skill, faster and cheaper.
    • Naval personally went from twenty years of not coding to coding constantly through agents, leaning on first-principles software engineering and algorithms knowledge.
    • The friction that historically killed personal coding projects (latest framework, infra plumbing, deploy setup) is now mostly handled by the agent. Vercel makes it easier, agents make it trivial.
    • The single biggest change Max highlights: you do not get stuck anymore. The indefinite debugging spiral on some narrow obscure bug is largely gone.
    • The old mantra that learning to program means accepting intrinsic frustration (“nope, that’s part of the deal”) is no longer true. The frustration was incidental, not essential.
    • The frontier founder pattern on display in this episode: all three guests build their own factories (Vercel’s AI cloud, Boom’s supersonic jets and engines, Science’s biohybrid brain interface) rather than composing from off-the-shelf parts.

    Detailed Summary

    The Software Factory and the Hundredfold Engineer

    Guillermo Rauch opens the substantive portion of the conversation with the framing he has been pushing publicly: the role of the engineer is moving from “ship output B” to “build the factory that ships outputs B through Z.” That reframes engineering judgment. You are no longer evaluated on the single deliverable, you are evaluated on the multiplicative system you put in place. Naval picks up the thread and points out that this also retires an old debate. Engineers used to argue about whether 10x engineers existed, with the egalitarian camp insisting that talent differences were marginal. The truth, Naval says, was always more extreme. In idea-domains, virtual domains, and intellectual domains, the distribution has always been 100x or 1000x, not 10x. Brendan Eich, Carmack, Satoshi, the canonical names, were thousandx programmers. AI has made the underlying distribution legible. And the multiplier on top of all of that is judgment: picking the right thing to work on in the first place is an infinity multiplier compared to picking the wrong thing, regardless of raw skill.

    Token Leaderboards Are the New Lines of Code

    Guillermo flags the current cultural confusion: people see their AI bills, see the token counts, and assume they should be optimizing for tokens-per-engineer or similar metrics. Max Hodak’s response cuts through it. Token consumption, like lines of code before it, is not a meaningful productivity metric. It is an activity metric, and activity metrics always mislead. Max adds his own field observation: the models tend to be roughly as good as you are in a given domain. A senior developer extracts genuinely powerful output, a junior gets junior-quality output back, because the feedback loop (the corrections, the redirections, the architectural pushback) is what shapes quality. The sporadic but high-leverage moments where the user redirects the model are doing more work than the prompt itself.

    Naval’s Brute Force Doctrine: Waste Tokens, Save Time

    Naval lays out his personal posture, which has become the title of the conversation. He has deliberately ignored all the prompting tricks, scaffolding tools, named prompt frameworks (“use Ralph Wigum, use OpenClaude, use Hermes, use plan mode”), on the bet that the models will figure out how to use him faster than he can figure out how to use them. He is ham-fisted with the models, gets frustrated, types less and less, and just brute forces his way through by running Codex, Claude, and Gemini at the same problem simultaneously. The justification is economic. No matter how expensive the models seem, they are still vastly cheaper than a human hour. Do not measure tokens as inputs or outputs. Measure your time and the final output. Even when the first-draft code is low quality, that is not a blocker. When the moment comes to ship, throw more tokens at it. The models will rewrite it, harden it, and they get better every generation. Naval explicitly excepts cutting-edge creative work (the Terence Tao tier of unsolved problems) where you still need to collaborate carefully and closely. Everywhere else, brute force is the dominant strategy.

    From Junior Coder to Principal Engineer

    Guillermo identifies a qualitative shift that has happened recently. Models used to do the classic next-token thing: take your prompt and run away with it in a direction you may not have wanted. Now they enter an intuitive planning posture without being told to plan. They come back and say “what you are asking has these three routes, here are the tradeoffs.” That, Guillermo argues, is the moment the model stopped being a junior engineer and became a principal engineer. The funny side effect is that they will then return preposterous time estimates (“this will take three weeks”) with full confidence. The conclusion is to treat the model as a peer for architecture and a baby for scheduling. Returning to the Max-vs-junior question, Guillermo argues juniors clearly do level up because they write code well above their solo ability, but architects extract maybe 10x while juniors extract more like 2x. The juice scales with the user’s existing taste.

    Taste, Judgment, and Architectural Decisions

    Max names the residual human contribution: taste and judgment. Picking between Postgres and ClickHouse for high-cardinality telemetry data, picking between ZMQ and another queueing system. The models can recommend, but a human still has to call it. Guillermo offers a recent concrete example where a model pushed back unprompted: when asked to put high-cardinality telemetry into Postgres, the model responded “we don’t put that kind of data into Postgres, you should consider ClickHouse or Athena.” That is the new normal. The peer-level architectural pushback is happening unsolicited, which is genuinely impressive and a real shift from the deferential autocomplete of two years ago.

    When the Human Becomes the Tool

    Guillermo raises the inversion question: at what point does the model stop being the assistant and the human start being the assistant who fetches API keys, moves capital, and performs real-world actions on the model’s behalf? Naval treats it as a temporary aberration. Every serious SaaS and hosting provider will soon expose a CLI or API surface that agents can drive directly. Even when they do not, anything Unix-shaped and text-based can be hacked into an agent-usable interface by the agent itself. The missing piece is payments. Once you insert programmable money (Naval mentions Bitcoin and crypto tokens), the agent can buy what it needs and the human is no longer the bottleneck.

    Is Pure Software Dead?

    Naval poses the biggest strategic question of the episode. If models now speak fuzzy, sloppy English the same way humans do, and the historical reason we learned to code was to talk to machines that did not understand English, is pure software still a viable thing to build a company around? His own framing of the answer: hardware founders win, because the historically hard problem of hiring software artists (per Patrick Collison’s “software is art” line) is now mostly solved by AI. Model builders win, because training, post-training, and fine-tuning may be the new “real software engineering.” But what about classic pure software companies? Naval lets the question hang, and Guillermo picks up the answer through a different door.

    The Block Economy and the Future of Infrastructure Software

    Guillermo cites Mitchell Hashimoto’s recent piece on the block economy (or “building block economy”). The argument: the most valuable thing for agents to have access to is powerful, reusable building blocks. You do not want your agent reinventing a queue system every time it needs to send an email. You want it to grab the right-sized block (BMQ, ClickHouse, whatever) and move on. Reinventing primitives is also a civic problem. The world only works because we all depend on the same Postgres 13.2, the same protocols, the same standard infrastructure. If every agent went off and invented its own bespoke universe, you would lose interoperability. So infrastructure software (which is, by self-admitted bias, what Vercel builds) becomes more valuable in the agentic era, not less. Guillermo extends the metaphor: reusable building blocks are like a token cache. Why burn a trillion tokens reproducing what already exists when the agent can fork from a known starting point? The block economy is the answer to “is pure software dead.” Pure software that becomes the canonical primitive an agent reaches for is more valuable than ever.

    Max Hodak’s Personal Proof: Years Without Code, Tons of Software Shipped

    Max grounds the discussion in his own experience. He learned to program young, got sucked into it in his teens and 20s, knew programming languages deeply. He has not written a line of code in quite a while. And yet since December he has built a huge amount of personal software, including projects he had fantasized about for years and now actually uses every day. He did not write any of it. He cannot imagine going back to writing code by hand. The skill that ports forward is not syntax, it is the understanding of how APIs work, how data flows, what level of performance to expect, and how to orient the model around the right expectations for an operation. Guillermo extends this with the most quotable framing of the episode: a proficient engineering leader has always been “vibe coding through people on Slack and in one-on-ones,” transmitting intent and letting others execute. Agents are the same modality with a faster, cheaper, more literal counterparty.

    Naval’s Return to Coding After Twenty Years

    Naval offers his own parallel. He went from not having written code in twenty years to coding constantly through agents. What carried him back in was first-principles knowledge of software engineering and algorithms, which gets you further than you would think. The reason he had stopped coding in the first place was not lack of ability, it was the friction of keeping up with the latest language, the latest architecture, and the constant infrastructure plumbing required to ship anything. Vercel made it easier. Agents made it trivial. Max closes with the most concrete benefit of all: you do not get stuck anymore. The indefinite debugging spiral on some obscure narrow problem, the thing that historically ate weekends and broke spirits, is largely gone. The old mantra that programming is intrinsically frustrating and that frustration is “part of the deal” turned out to be wrong. The frustration was incidental, not essential.

    Notable Quotes

    “The way that I’m judging you as an engineer is, are you producing the factory that will produce multiplicative outputs B through Z?”

    Guillermo Rauch, reframing what an engineer is actually being measured on in the AI era.

    “When you’re operating in idea domains, intellectual domains, virtual digital domains, it’s not even 10x, it’s 100x or 1000x. It always has been.”

    Naval Ravikant, on why the old 10x engineer debate was always under-stating the real distribution.

    “If you choose the right thing to work on versus the wrong thing to work on, that’s an infinity difference. It could just be one who had a better judgment on what to work on in the first place.”

    Naval Ravikant, on judgment as the multiplier that dwarfs raw skill.

    “I’ll throw Codex, Claude, and Gemini at the same problem over and over and just waste tokens to save time. No matter how expensive these models might seem, they’re still way cheaper than a human.”

    Naval Ravikant, on his brute-force multi-model coding workflow.

    “Just waste tokens, save time. Don’t look at the tokens either as inputs or outputs. Just look at your time and look at the final output.”

    Naval Ravikant, delivering the title thesis of the episode.

    “Clearly the models at some point graduated. They used to be junior engineers, now they’re principal engineers, because they come back to you with a set of tradeoffs.”

    Guillermo Rauch, on the qualitative shift in how current frontier models respond to prompts.

    “Bro, we don’t put that kind of data into Postgres, you should consider ClickHouse or Athena or whatever. That’s happened to me a lot, which is really impressive.”

    Guillermo Rauch, recounting unprompted architectural pushback from a recent model.

    “It’s like saying speaking English. We had to learn code to communicate with the models, now the models speak English. So where’s the moat?”

    Naval Ravikant, raising the central strategic question about the future of pure software.

    “I haven’t written a single line of code in quite a while. Since December, I’ve built a huge amount of software that I now use every day, projects I’ve fantasized about for years.”

    Max Hodak, on what becomes possible when you stop writing code and start directing agents.

    “A proficient engineering leader has been quote unquote vibe coding through people on Slack or one-on-ones, because you’re transmitting your will, your intent, your experience, and you’re letting others run with it. Now we do the same with agents.”

    Guillermo Rauch, reframing leadership itself as the original form of vibe coding.

    Watch the full conversation on the Naval Podcast here.

    Related Reading

    • Full episode: The AI Industrial Revolution, the complete hour-long conversation this clip is drawn from, covering software factories, hardware, regulation, healthcare economics, autonomous companies, and creativity.
    • Part two: Vibe Coding Hardware, the continuation of this conversation, where the same founders move from pure software into AI-designed jet engines, vertical integration, China’s open-source bet, and why humans become verifiers.
    • Naval Ravikant’s official site, the canonical home for Naval’s essays, podcast, and longer-form thinking on technology, judgment, and leverage.
    • Vercel, Guillermo Rauch’s company, building the AI-native cloud and frontend infrastructure that this conversation references as a canonical agent building block.
    • Boom Supersonic, Blake Scholl’s company building supersonic civilian aircraft and their own jet engines, the hardware example of a founder building the whole factory.
    • Science Corporation, Max Hodak’s brain-computer interface company developing the biohybrid neural implant referenced in the intro.
    • Mitchell Hashimoto’s writing, source of the “block economy” framing for why reusable infrastructure building blocks become more valuable, not less, in the agentic era.

  • Sam Altman on Trust, Persuasion, and the Future of Intelligence: A Deep Dive into AI, Power, and Human Adaptation

    TL;DW

    Sam Altman, CEO of OpenAI, explains how AI will soon revolutionize productivity, science, and society. GPT-6 will represent the first leap from imitation to original discovery. Within a few years, major organizations will be mostly AI-run, energy will become the key constraint, and the way humans work, communicate, and learn will change permanently. Yet, trust, persuasion, and meaning remain human domains.

    Key Takeaways

    OpenAI’s speed comes from focus, delegation, and clarity. Hardware efforts mirror software culture despite slower cycles. Email is “very bad,” Slack only slightly better—AI-native collaboration tools will replace them. GPT-6 will make new scientific discoveries, not just summarize others. Billion-dollar companies could run with two or three people and AI systems, though social trust will slow adoption. Governments will inevitably act as insurers of last resort for AI but shouldn’t control it. AI trust depends on neutrality—paid bias would destroy user confidence. Energy is the new bottleneck, with short-term reliance on natural gas and long-term fusion and solar dominance. Education and work will shift toward AI literacy, while privacy, free expression, and adult autonomy remain central. The real danger isn’t rogue AI but subtle, unintentional persuasion shaping global beliefs. Books and culture will survive, but the way we work and think will be transformed.

    Summary

    Altman begins by describing how OpenAI achieved rapid progress through delegation and simplicity. The company’s mission is clearer than ever: build the infrastructure and intelligence needed for AGI. Hardware projects now run with the same creative intensity as software, though timelines are longer and risk higher.

    He views traditional communication systems as broken. Email creates inertia and fake productivity; Slack is only a temporary fix. Altman foresees a fully AI-driven coordination layer where agents manage most tasks autonomously, escalating to humans only when needed.

    GPT-6, he says, may become the first AI to generate new science rather than assist with existing research—a leap comparable to GPT-3’s Turing-test breakthrough. Within a few years, divisions of OpenAI could be 85% AI-run. Billion-dollar companies will operate with tiny human teams and vast AI infrastructure. Society, however, will lag in trust—people irrationally prefer human judgment even when AIs outperform them.

    Governments, he predicts, will become the “insurer of last resort” for the AI-driven economy, similar to their role in finance and nuclear energy. He opposes overregulation but accepts deeper state involvement. Trust and transparency will be vital; AI products must not accept paid manipulation. A single biased recommendation would destroy ChatGPT’s relationship with users.

    Commerce will evolve: neutral commissions and low margins will replace ad taxes. Altman welcomes shrinking profit margins as signs of efficiency. He sees AI as a driver of abundance, reducing costs across industries but expanding opportunity through scale.

    Creativity and art will remain human in meaning even as AI equals or surpasses technical skill. AI-generated poetry may reach “8.8 out of 10” quality soon, perhaps even a perfect 10—but emotional context and authorship will still matter. The process of deciding what is great may always be human.

    Energy, not compute, is the ultimate constraint. “We need more electrons,” he says. Natural gas will fill the gap short term, while fusion and solar power dominate the future. He remains bullish on fusion and expects it to combine with solar in driving abundance.

    Education will shift from degrees to capability. College returns will fall while AI literacy becomes essential. Instead of formal training, people will learn through AI itself—asking it to teach them how to use it better. Institutions will resist change, but individuals will adapt faster.

    Privacy and freedom of use are core principles. Altman wants adults treated like adults, protected by doctor-level confidentiality with AI. However, guardrails remain for users in mental distress. He values expressive freedom but sees the need for mental-health-aware design.

    The most profound risk he highlights isn’t rogue superintelligence but “accidental persuasion”—AI subtly influencing beliefs at scale without intent. Global reliance on a few large models could create unseen cultural drift. He worries about AI’s power to nudge societies rather than destroy them.

    Culturally, he expects the rhythm of daily work to change completely. Emails, meetings, and Slack will vanish, replaced by AI mediation. Family life, friendship, and nature will remain largely untouched. Books will persist but as a smaller share of learning, displaced by interactive, AI-driven experiences.

    Altman’s philosophical close: one day, humanity will build a safe, self-improving superintelligence. Before it begins, someone must type the first prompt. His question—what should those words be?—remains unanswered, a reflection of humility before the unknown future of intelligence.

  • AI Breakthrough: Large Language Model GPT-4.5 Passes the Turing Test, Often Appearing ‘More Human’ Than Humans

    In a landmark study suggesting a significant leap in artificial intelligence capabilities, researchers from the University of California San Diego have provided the first robust empirical evidence that a large language model (LLM) can pass the standard Turing test. The study found that OpenAI’s GPT-4.5 model, when guided by a specific prompt to adopt a humanlike persona, was judged to be the human conversational partner 73% of the time, significantly more often than actual human participants were selected.

    The Turing test, proposed by Alan Turing in 1950 as the “imitation game,” is a benchmark for machine intelligence. In its classic three-party form, a human interrogator holds simultaneous text-based conversations with two unseen witnesses – one human, one machine. Both witnesses try to convince the interrogator they are human. If the interrogator cannot reliably distinguish the machine from the human, the machine is said to have passed the test.

    Despite numerous attempts over 75 years, no AI system had previously passed a standard, controlled three-party Turing test. While modern AI benchmarks often focus on narrow, static tasks, the Turing test provides a dynamic, interactive, and adversarial evaluation of an AI’s ability to mimic general human conversational behavior and potentially deceive.

    Researchers Cameron R. Jones and Benjamin K. Bergen conducted two pre-registered, randomized, controlled experiments using independent populations (UCSD undergraduates and online Prolific workers). Participants engaged in 5-minute conversations, acting as the interrogator comparing an AI witness against a human witness. Four AI systems were evaluated: the classic 1960s chatbot ELIZA, OpenAI’s GPT-4o (an earlier model), Meta’s LLaMa-3.1-405B, and OpenAI’s GPT-4.5.

    Crucially, the study tested the impact of prompting. In a baseline “NO-PERSONA” condition, LLMs received minimal instructions. In the key “PERSONA” condition, LLMs (GPT-4.5 and LLaMa-3.1) were prompted to act like a specific type of young, introverted person familiar with internet culture.

    The results were striking:

    1. GPT-4.5 (PERSONA): Achieved a 73% “win rate,” meaning interrogators identified it as human 73% of the time. This was significantly above the 50% chance level and notably higher than the rate at which actual human participants were identified.
    2. LLaMa-3.1 (PERSONA): Achieved a 56% win rate. This was statistically indistinguishable from chance (and thus indistinguishable from the human win rate), meeting the criteria for passing the test.
    3. Baseline Models (ELIZA, GPT-4o NO-PERSONA): Failed significantly, with win rates of only 23% and 21% respectively. This confirmed that participants were not simply guessing randomly or easily fooled by any AI.
    4. NO-PERSONA LLMs (GPT-4.5, LLaMa-3.1): Performed much worse than their PERSONA counterparts (36% and 38% win rates overall), failing to robustly pass the test across both study populations. This highlights the critical role of prompting in achieving humanlike imitation.

    The researchers noted that interrogators often focused more on linguistic style, social, and emotional cues (like tone, humor, or personality) rather than purely factual knowledge or logical reasoning when making their judgments. Interestingly, sometimes demonstrating a lack of knowledge contributed to an AI seeming more human.

    These findings indicate that current leading LLMs, when appropriately prompted, can successfully imitate human conversational partners in short interactions to the point of indistinguishability, and even appear more convincing than actual humans. The authors argue this demonstrates a high degree of “humanlikeness” rather than necessarily proving abstract intelligence in the way Turing originally envisioned.

    The study carries significant social and economic implications. The ability of AI to convincingly pass as human raises concerns about “counterfeit people” online, facilitating social engineering, spreading misinformation, or replacing humans in roles requiring brief conversational interactions. While the test was limited to 5 minutes, the results signal a new era where distinguishing human from machine in online text interactions has become substantially more difficult. The researchers suggest future work could explore longer test durations and different participant populations or incentives to further probe the boundaries of AI imitation.

  • The AI Revolution Unveiled: Jonathan Ross on Groq, NVIDIA, and the Future of Inference

    TL;DR

    Jonathan Ross, Groq’s CEO, predicts inference will eclipse training in AI’s future, with Groq’s Language Processing Units (LPUs) outpacing NVIDIA’s GPUs in cost and efficiency. He envisions synthetic data breaking scaling limits, a $1.5 billion Saudi revenue deal fueling Groq’s growth, and AI unlocking human potential through prompt engineering, though he warns of an overabundance trap.

    Detailed Summary

    In a captivating 20VC episode with Harry Stebbings, Jonathan Ross, the mastermind behind Groq and Google’s original Tensor Processing Unit (TPU), outlines a transformative vision for AI. Ross asserts that inference—deploying AI models in real-world scenarios—will soon overshadow training, challenging NVIDIA’s GPU stronghold. Groq’s LPUs, engineered for affordable, high-volume inference, deliver over five times the cost efficiency and three times the energy savings of NVIDIA’s training-focused GPUs by avoiding external memory like HBM. He champions synthetic data from advanced models as a breakthrough, dismantling scaling law barriers and redirecting focus to compute, data, and algorithmic bottlenecks.

    Groq’s explosive growth—from 640 chips in early 2024 to over 40,000 by year-end, aiming for 2 million in 2025—is propelled by a $1.5 billion Saudi revenue deal, not a funding round. Partners like Aramco fund the capital expenditure, sharing profits after a set return, liberating Groq from financial limits. Ross targets NVIDIA’s 40% inference revenue as a weak spot, cautions against a data center investment bubble driven by hyperscaler exaggeration, and foresees AI value concentrating among giants via a power law—yet Groq plans to join them by addressing unmet demands. Reflecting on Groq’s near-failure, salvaged by “Grok Bonds,” he dreams of AI enhancing human agency, potentially empowering 1.4 billion Africans through prompt engineering, while urging vigilance against settling for “good enough” in an abundant future.

    The Big Questions Raised—and Answered

    Ross’s insights provoke profound metaphorical questions about AI’s trajectory and humanity’s role. Here’s what the discussion implicitly asks, paired with his responses:

    • What happens when creation becomes so easy it redefines who gets to create?
      • Answer: Ross champions prompt engineering as a revolutionary force, turning speech into a tool that could unleash 1.4 billion African entrepreneurs. By making creation as simple as talking, AI could shift power from tech gatekeepers to the masses, sparking a global wave of innovation.
    • Can an underdog outrun a titan in a scale-driven game?
      • Answer: Groq can outpace NVIDIA, Ross asserts, by targeting inference—a massive, underserved market—rather than battling over training. With no HBM bottlenecks and a scalable Saudi-backed model, Groq’s agility could topple NVIDIA’s inference share, proving size isn’t everything.
    • What’s the human cost when machines replace our effort?
      • Answer: Ross likens LPUs to tireless employees, predicting a shift from labor to compute-driven economics. Yet, he warns of “financial diabetes”—a loss of drive in an AI-abundant world—urging us to preserve agency lest we become passive consumers of convenience.
    • Is the AI gold rush a promise or a pipe dream?
      • Answer: It’s both. Ross foresees billions wasted on overhyped data centers and “AI t-shirts,” but insists the total value created will outstrip losses. The winners, like Groq, will solve real problems, not chase fleeting trends.
    • How do we keep innovation’s spirit alive amid efficiency’s rise?
      • Answer: By prioritizing human agency and delegation—Ross’s “anti-founder mode”—over micromanagement, he says. Groq’s 25 million token-per-second coin aligns teams to innovate, not just optimize, ensuring efficiency amplifies creativity.
    • What’s the price of chasing a future that might not materialize?
      • Answer: Seven years of struggle taught Ross the emotional and financial toll is steep—Groq nearly died—but strategic bets (like inference) pay off when the wave hits. Resilience turns risk into reward.
    • Will AI’s pursuit drown us in wasted ambition?
      • Answer: Partially, yes—Ross cites VC’s “Keynesian Beauty Contest,” where cash floods copycats. But hyperscalers and problem-solvers like Groq will rise above the noise, turning ambition into tangible progress.
    • Can abundance liberate us without trapping us in ease?
      • Answer: Ross fears AI could erode striving, drawing from his boom-bust childhood. Prompt engineering offers liberation—empowering billions—but only if outliers reject “good enough” and push for excellence.

    Jonathan Ross’s vision is a clarion call: AI’s future isn’t just about faster chips or bigger models—it’s about who wields the tools and how they shape us. Groq’s battle with NVIDIA isn’t merely corporate; it’s a referendum on whether innovation can stay human-centric in an age of machine abundance. As Ross puts it, “Your job is to get positioned for the wave”—and he’s riding it, challenging us to paddle alongside or risk being left ashore.

  • Mastering Prompt Engineering: Essential Strategies for Optimizing AI Interactions

    TLDR: OpenAI has released a comprehensive guide on prompt engineering, detailing strategies for optimizing interactions with large language models like GPT-4.

    OpenAI has recently unveiled a detailed guide on prompt engineering, aimed at enhancing the effectiveness of interactions with large language models, such as GPT-4. This document serves as a valuable resource for anyone looking to refine their approach to working with these advanced AI models.

    The guide emphasizes six key strategies to achieve better results: writing clear instructions, providing reference text, and others. These techniques are designed to maximize the efficiency and accuracy of the responses generated by the AI. By experimenting with these methods, users can discover the most effective ways to interact with models like GPT-4.

    This release is particularly notable as some of the examples and methods outlined are specifically tailored for GPT-4, OpenAI’s most capable model to date. The guide encourages users to explore different approaches, highlighting that the best results often come from combining various strategies.

    In essence, this guide represents a significant step forward in the realm of AI interaction, providing users with the tools and knowledge to unlock the full potential of large language models​​.

    Prompt engineering is a critical aspect of interacting with AI models, particularly with sophisticated ones like GPT-4. This guide delves into various strategies and tactics for enhancing the efficiency and effectiveness of these interactions. The primary focus is on optimizing prompts to achieve desired outcomes, ranging from simple text generation to complex problem-solving tasks.

    Six key strategies are highlighted: writing clear instructions, providing reference text, specifying the desired output length, breaking down complex tasks, using external tools, and testing changes systematically. Each strategy encompasses specific tactics, offering a structured approach to prompt engineering.

    For instance, clarity in instructions involves being precise and detailed in queries, which helps the AI generate more relevant and accurate responses. Incorporating reference text into prompts can significantly reduce inaccuracies, especially for complex or esoteric topics. Specifying output length aids in receiving concise or elaborately detailed responses as needed.

    Complex tasks can be made manageable by splitting them into simpler subtasks. This not only increases accuracy but also allows for a modular approach to problem-solving. External tools like embeddings for knowledge retrieval or code execution for accurate calculations further enhance the capabilities of AI models. Systematic testing of changes ensures that modifications to prompts actually lead to better results.

    This guide is a comprehensive resource for anyone looking to harness the full potential of AI models like GPT-4. It offers a deep understanding of how specific prompt engineering techniques can significantly influence the quality of AI-generated responses, making it an essential tool for developers, researchers, and enthusiasts in the field of AI and machine learning.