PJFP.com

Pursuit of Joy, Fulfillment, and Purpose

Tag: large language models

  • Jensen Huang at Stanford CS153 Frontier Systems on Co-Design, Agentic Computing, Vera Rubin, Open Models, and the Million-X Decade That Reshaped AI Infrastructure

    https://www.youtube.com/watch?v=tsQB0n0YV3k

    NVIDIA CEO Jensen Huang returned to Stanford for the CS153 Frontier Systems class (the room nicknamed itself “AI Coachella”) to lay out, in raw form, how he thinks about the computer being reinvented for the first time in over sixty years. Across roughly seventy minutes of student questions he walks through the codesign philosophy that gave NVIDIA a million-x decade, the architectural through-line from Hopper to Grace Blackwell to Vera Rubin to Feynman, the case for open source foundation models, the realities of tokens per watt and MFU, energy demand running a thousand times higher, the China and export-control debate, and his own biggest strategic mistakes. Watch the full conversation on YouTube.

    TLDW

    Huang argues every layer of computing has changed: the programming model, the system architecture, the deployment pattern, the economics. Co-design across CPUs, GPUs, networking, storage, switches and compilers gave NVIDIA roughly a million-x speed-up over ten years versus the ten-x Moore’s Law era, and that headroom is what let researchers say “just train on the whole internet.” Hopper was built for pre-training, Grace Blackwell NVLink72 for inference and reasoning (50x over Hopper in two years), Vera Rubin is built for agents that load long memory, call tools and need a low-latency single-threaded CPU bolted directly to the GPU, and Feynman extends that to swarms of agents that spawn sub-agents. Open weights matter because safety, sovereignty (230-plus languages no one else will fund) and domain models for biology, autonomy, robotics and climate need a foundation that NVIDIA is willing to seed. Compute is not really the scarce resource (Huang says place the order and the chips ship), the broken thing is institutional budgeting that can’t put a billion dollars into a shared university supercomputer. Energy demand is heading a thousand times higher and this is finally the moment market forces alone will fund sustainable generation. On geopolitics he rejects the GPUs-as-atomic-bombs framing and warns America will end up like its telecom industry if it cedes two thirds of the world. On career he advises seeking suffering on purpose. On strategy he says observe, reason from first principles, build a mental model, work backwards, minimize opportunity cost, maximize optionality.

    Key Takeaways

    • The computing model has been substantially unchanged since the IBM System 360, sixty-plus years ago. Huang’s first computer architecture book was the System 360 manual. AI is the first true reinvention.
    • Old computing was pre-recorded retrieval. New computing is generated, contextually aware and continuous. Cloud was on-demand. Agentic systems run continuously.
    • Codesign is NVIDIA’s central thesis. Inherited from the Hennessy and Patterson RISC era at Stanford, extended across CPUs, GPUs, networking, switches, storage, compilers and frameworks all optimized together.
    • The result of full-stack codesign: roughly 1,000,000x faster compute over ten years, versus a generous 10x to 100x for Moore’s Law in the same period. Dennard scaling effectively ended a decade ago.
    • That million-x speed-up is what unlocked “train on all of the internet” as a realistic AI strategy.
    • After GPT, Huang says it was obvious thinking was next. Reasoning is just generating tokens consumed internally, then using tools is generating tokens consumed externally. Agentic systems followed predictably.
    • Education needs AI baked into the curriculum, not just taught as a subject. Pre-recorded textbooks cannot keep pace with knowledge being generated in real time.
    • Huang says he cannot learn anymore without AI. He has the AI read the paper, then read every related paper, then become a dedicated researcher he can interrogate.
    • Mead and Conway and the first-principles methodology of semiconductor design are still worth learning even though most of the scaling tricks have been exhausted.
    • NVIDIA itself is one of the largest consumers of Anthropic and OpenAI tokens in the world. One hundred percent of NVIDIA engineers are now agentically supported. Huang recommends Claude and similar tools by name and says open-source downloads will not match the integrated product harness.
    • NVIDIA still invests heavily in open foundation models because language and intelligence represent the codification of human knowledge. Five pillars: Nemotron (language), BioNeMo (biology), Alphamayo (autonomous vehicles), Groot (humanoid robotics) and a climate science model (mesoscale multiphysics).
    • Sovereign language models matter. Roughly 230 world languages will never be a top priority for a commercial frontier lab. Nemotron is near-frontier and fully fine-tunable so any country can adapt it.
    • Safety and security require open weights. You cannot defend against or audit a black box. Transparent systems let researchers interrogate models and let defenders deploy swarms.
    • The future of cyber defense is not bigger-model-versus-bigger-model. It is trillions of cheap fast small models like Nemotron Nano surrounding the threat.
    • Domain models fuse language priors with world models. Alphamayo learned to drive safely on a few million miles instead of billions because it can reason like a human about the road.
    • MFU (Model Flops Utilization) is a misleading metric. Huang says he wants low MFU, because that means he over-provisioned every resource and never gets pinned by Amdahl’s law during a spike.
    • The xAI Memphis cluster running at 11 percent MFU is not necessarily a failure mode. In disaggregated prefill plus decode inference you can deliver very high tokens per watt with very low MFU.
    • The right metric is performance, ultimately tokens per watt as a proxy for intelligence per watt, and even that needs adjustment because not all tokens are equal. Coding tokens are worth more than other tokens.
    • Hopper was designed for pre-training. NVIDIA chose to build multi-billion-dollar systems when the largest existing scientific supercomputer cost $350 million, with no proven customer base. It worked.
    • Grace Blackwell NVLink72 was designed for inference, especially the high-memory-bandwidth decode phase. It is the world’s first rack-scale computer and delivered a 50x speed-up over Hopper in two years, against an expected 2x from Moore’s Law.
    • Vera Rubin is designed for agents. Long-term memory wired into storage and into the GPU fabric, working memory, heavy tool use, and Vera, a CPU optimized for low-latency multi-core single-threaded code so a multi-billion-dollar GPU system does not stall waiting on a slow tool call.
    • Feynman is being shaped for swarms of agents with sub-agents and sub-sub-agents, a recursive software topology that demands a new compute pattern.
    • Tokens per watt improved 50x in one generation. Compounding energy efficiency is the lever NVIDIA controls directly.
    • Total compute energy demand is heading roughly a thousand times higher than today, possibly two orders of magnitude beyond that. Huang says he would not be surprised if the estimate is low.
    • For the first time in history, market forces alone are enough to fund solar, nuclear and grid upgrades. Government subsidies are no longer required to make sustainable energy investment rational.
    • Copper interconnect is becoming a bottleneck. Photonics is moving from optional to structural inside racks and across them.
    • Comparing NVIDIA GPUs to atomic bombs, Huang says, is a stupid analogy. A billion people use NVIDIA GPUs. He advocates them to his family. He does not advocate atomic bombs to anyone.
    • If the United States cedes two thirds of the global market to competitors on policy grounds, the American technology industry will end up like American telecommunications, which was policied out of existence.
    • Huang directly rejects AI doom-by-singularity narratives. It is not true that we have no idea how these systems work. It is not true that the technology becomes infinitely powerful in a nanosecond. He calls the rhetoric irresponsible and harmful to the field students are about to enter.
    • On Stanford specifically: if the university president places an order, NVIDIA will deliver the chips. The bottleneck is that no university department has a billion-dollar compute budget because budgeting is fragmented across grants. Stanford’s $40 billion endowment is more than enough to fix that.
    • “It’s Stanford’s fault” is meant as empowerment. If something is your fault, you can solve it.
    • Career advice: do not optimize purely for passion. Most people do not yet know what they love. Pick the job in front of you and do it as well as possible. Even as CEO, Huang says, 90 percent of the work is hard and he suffers through it.
    • Suffering on purpose builds the muscle of resilience. When the company, the team or the family needs you to be tough, that muscle has to already exist.
    • NVIDIA’s first generation of products was technically wrong in nearly every dimension: curved surfaces instead of triangles, no Z-buffer, forward instead of inverse texture mapping, no floating point. The strategic recovery, not the technology, taught Huang the lessons that have lasted decades.
    • The biggest clean strategic mistake Huang names is the move into mobile chips (Tegra). It grew to a billion dollars then went to zero when Qualcomm’s modem dominance shut NVIDIA out of the 3G to 4G transition. The recovery into automotive and robotics (the Thor chip is the great great great grandson of that mobile lineage) was real, but Huang refuses to rationalize the original choice.
    • Forecasting framework: observe, reason from first principles, ask “so what” and “what next” until you have a mental model of the future, place your company inside that model, then work backwards while minimizing opportunity cost and maximizing optionality.
    • Best part of the CEO job: living at the intersection of vision, strategy and execution surrounded by people capable enough to make ambitious visions real. Worst part: the responsibility for everyone who joined the spaceship, especially in the near-death moments NVIDIA had four or five times early on.
    • Underrated insider note: Huang’s first apple pie with cheese, first hot fudge sandwich and first milkshake all happened at Denny’s. The Superbird, the fried chicken and a custom Superbird-style ham and cheese with tomato and mustard are his order.

    Detailed Summary

    Computing reinvented from the ground up

    Huang frames the moment as the first true rewrite of the computer in sixty-plus years. From the IBM System 360 forward, the mental model of writing code, running code, taking a computer to market and reasoning about applications stayed roughly constant. AI changes the programming model itself. Software is no longer a compiled binary running deterministically on a CPU. It is a neural network running on a GPU producing generated, contextual, real-time output. That cascades into how companies are organized, what tools developers use, what the network and storage stack look like, and what an application is even allowed to do. Robo-taxis, he notes, are an application no one would have attempted before deep learning unlocked perception.

    Codesign and the million-x decade

    Codesign is the philosophical center of the talk. Huang traces it to the RISC work of John Hennessy at Stanford, where simpler instruction sets won by being co-designed with the compiler rather than maximally optimized in isolation. NVIDIA extends the principle across every layer simultaneously: GPU architecture, CPU architecture, NVLink and NVSwitch fabrics, photonic interconnects, networking silicon, storage paths, CUDA libraries, frameworks and ultimately the model design. The numbers Huang gives are arresting. Moore’s Law in its prime delivered roughly 100x per decade. By the time Dennard scaling broke, real-world gains had compressed to roughly 10x. NVIDIA’s codesigned stack delivered between 100,000x and 1,000,000x over the same ten-year window. That non-linear speed-up is, in Huang’s telling, the precondition for modern AI: it is what allowed researchers to stop curating training sets and just feed the entire internet to the model.

    Education has to fuse first principles with AI tools

    Asked how curriculum should evolve, Huang argues AI must be integrated into the learning process, not just taught about. He recalls Hennessy writing his textbook by hand a chapter a week while Huang was a student, and says pre-recorded textbooks cannot keep up with the rate at which AI generates new knowledge. He describes his own learning workflow: hand the paper to an AI, then have it read the entire surrounding literature, then treat the AI as a dedicated researcher who can be interrogated. At the same time he defends the classics. Mead and Conway are still the foundation. Most modern semiconductor scaling tricks have been exhausted, but knowing where the field came from sharpens judgment when designing what comes next.

    Open source and the five domain pillars

    Huang gives one of the most detailed public accounts of why NVIDIA invests so heavily in open foundation models even while being a top customer of closed labs. He recommends Claude and OpenAI by name for production coding work, and says 100 percent of NVIDIA engineers are now agentically supported. The open-weights case rests on three legs. First, language is the codification of intelligence, and there are at least 230 languages that no commercial lab will ever prioritize. Nemotron is built near frontier and released so any country or community can fine-tune it. Second, the same representation-learning approach has to be replicated in domains where the data is not internet text, so NVIDIA seeded BioNeMo for biology, Alphamayo for autonomy, Groot for humanoid robotics and a climate model for mesoscale multiphysics. The economics of those fields would never produce a foundation model on their own. Third, safety and security require transparency. A black box cannot be defended or audited, and the future of cyber defense is not bigger-model-versus-bigger-model but swarms of cheap fast small models like Nemotron Nano surrounding the threat.

    MFU is the wrong metric, tokens per watt is closer

    A student raises the leaked memo that the xAI Memphis cluster is running at 11 percent Model Flops Utilization. Huang flips the framing. He says he would rather be at low MFU all the time, because that means he over-provisioned flops, memory bandwidth, memory capacity and network capacity. Bottlenecks shift constantly, so over-provisioning across every dimension is what lets the system absorb a spike without getting pinned by Amdahl’s law. In disaggregated inference, where prefill and decode are physically separated and decode is bandwidth-bound rather than flop-bound, NVLink72 can deliver extremely high tokens per watt while reporting very low MFU. Huang argues the right framing is performance, and ultimately tokens per watt as a rough proxy for intelligence per watt, adjusted for the fact that not all tokens are equal. A coding token is worth more than a generic token.

    Hopper, Grace Blackwell NVLink72, Vera Rubin, Feynman

    Huang gives the clearest public framing of NVIDIA’s roadmap as a sequence of architectural answers to evolving compute patterns. Hopper was built for pre-training, at a moment when NVIDIA chose to build multi-billion-dollar machines while the largest scientific supercomputer in the world cost $350 million and the marketplace for such systems was, on paper, zero. Grace Blackwell NVLink72 was the answer to inference and reasoning: a rack-scale computer that ganged 72 GPUs together because decode needs aggregate memory bandwidth far beyond a single chip. The generation-over-generation speed-up was 50x in two years, twenty-five times what Moore’s Law would have delivered. Vera Rubin is being built explicitly for agents. Agents load long-term memory from storage that has to be wired directly into the GPU fabric, they use working memory, they call tools that run on a CPU, and they wait. So the CPU has to be Vera, optimized for low-latency single-threaded code, because the multi-billion-dollar GPU system cannot afford to idle waiting on a slow tool call. Feynman extends the pattern to swarms of agents with sub-agents and sub-sub-agents, a recursive software topology that will demand its own compute pattern.

    Energy demand and the grid

    Huang’s energy projection is one of the most aggressive numbers in the talk. NVIDIA can compound tokens per watt by 50x per generation through codesign, but the total compute demand is heading roughly a thousand times higher, and Huang says he would not be surprised if the real figure is one or two orders of magnitude beyond that. The reason is structural: future computing is generative and continuous, not pre-recorded and on-demand. The good news, he argues, is that this is the best moment in the history of humanity to invest in sustainable generation. Market forces alone are now sufficient to fund solar, nuclear and grid upgrades. Government subsidies are no longer required to make the math work.

    Adversarial countries, export controls and the telecom warning

    This is the segment where Huang is visibly fired up. He attacks the GPUs-as-atomic-bombs framing on its face. NVIDIA GPUs power medical imaging, video games and soy sauce delivery. A billion people use them. He advocates them to his family. The analogy collapses at the first comparison. He attacks the second framing, that American companies should not compete abroad because they will lose anyway, as a self-fulfilling defeat. Competition makes the company better. The third framing, that depriving the rest of the world of general-purpose computing benefits the United States, also fails on first principles: it benefits one or two American companies at the cost of an entire industry. The cautionary parallel is telecommunications. The United States once had a leading position in telecom fundamental technology and policied itself out of it. Huang’s worry, voiced explicitly to a room of CS students, is that they will graduate into a shell of a computer industry if the same path is repeated.

    AI doom and rational optimism

    In the same arc Huang rejects the science-fiction framing of AI as a singularity that arrives suddenly on a Wednesday at 7pm and ends civilization. He calls those claims irresponsible, says they are not true, and points out that the people advancing them are believed by audiences who then make policy on that basis. It is not true that no one understands how these systems work. It is not true that intelligence becomes infinitely powerful instantaneously. It is not true that there is no defense. His framing, which the host echoes as “rational optimism,” is that the goal is to create a future where people care about computers because the technology students are learning is worth mastering.

    Stanford’s compute problem is Stanford’s fault

    A student presses on the scarcity of compute for independent researchers, startups and universities inside the United States. Huang’s answer is sharp: there is no shortage. Place the order and the chips will arrive. The actual broken thing is institutional. University grants are fragmented across departments. No researcher can raise enough on a single grant to fund a billion-dollar shared cluster, and no one shares. He compares it to showing up at the grocery store demanding a billion dollars of tomatoes today. The solution is planning, aggregation and a campus-scale supercomputer, the way Stanford once built the linear accelerator. The endowment is $40 billion. Pulling a billion off it, contracting cloud capacity and giving every student and researcher AI supercomputer access is, in Huang’s view, obviously doable. When he says “it is Stanford’s fault” the host laughs, but Huang clarifies: if it is your fault you have the power to fix it.

    Career, suffering and resilience

    Asked how a CS student should spend the next few years, Huang pushes back on the standard “follow your passion” advice. Most people do not know what they love yet, because no one knows what they do not know. The bar of demanding joy from every working day is too high. Whatever the job is, do it as well as you can. Even as CEO of NVIDIA he says he genuinely loves about 10 percent of his work. The other 90 percent is hard and he suffers through it. He recommends suffering on purpose, because resilience is a muscle that only builds under load, and when the company, the team or the family needs that muscle, it has to already exist. Earlier in his life that meant cleaning toilets and busing tables at Denny’s. He does it today running a multi-trillion-dollar company.

    The biggest mistakes

    Huang separates technical mistakes from strategic mistakes. NVIDIA’s first generation of products was technically wrong in almost every way: curved surfaces instead of triangles, no Z-buffer, forward instead of inverse texture mapping, no floating point inside. The company wasted two and a half years. But the strategic genius of the recovery, the reading of the market, the conservation of resources and the reapplication of talent, is what taught him strategy. The clean strategic mistake he names is mobile. NVIDIA’s Tegra line grew to a billion dollars of revenue and then collapsed to zero when Qualcomm’s modem dominance locked NVIDIA out of the 3G to 4G transition. Huang explicitly refuses the comforting rationalization that the Tegra effort fed the Thor automotive chip (“Thor is the great great great grandson”). The original decision, he says, was a waste of time. The lesson is to think one or two clicks further about whether a market is structurally winnable before committing the company.

    Forecasting under fog of war

    The final substantive exchange is on forecasting. Huang’s method has four steps. Observe what is actually happening (AlexNet crushing two decades of computer vision research in one shot, GPT producing reasoning by token generation). Reason from first principles about why it works. Ask “so what” and “what next” recursively until a mental model of the future emerges. Place the company inside that future and work backwards. Crucially, expect to be partly wrong. Some outcomes will absolutely happen, some will likely happen, some might happen, and the strategy has to be robust across that distribution. The real cost of any strategic choice is the opportunity cost of the alternatives you did not take, so the discipline is to minimize that cost and maximize optionality while letting the journey itself pay for the journey.

    Thoughts

    The most useful thing in this conversation is the explicit architectural mapping of compute patterns to chip generations. Hopper for pre-training. Grace Blackwell NVLink72 for inference, because decode is bandwidth-bound and a single chip cannot supply it. Vera Rubin for agents, because tool calls stall multi-billion-dollar GPU systems and so the CPU has to be optimized for low-latency single-threaded code. Feynman for swarms. That sequence is not marketing. It is a falsifiable thesis about where the bottleneck moves next, and every other infrastructure company should be measuring themselves against it. If Huang is right that swarms of sub-agents are the next dominant pattern, then the design pressure shifts from raw flops to fabric topology, memory hierarchy and storage-to-GPU latency. That has implications for everyone downstream, including the hyperscalers building competing accelerators.

    The MFU section is the most intellectually generous moment in the talk. The instinct in the AI ops community has been to chase MFU as if it were a virtue. Huang argues, persuasively, that low MFU is consistent with high tokens per watt in a disaggregated inference setup, and that bottlenecks rotate fast enough that over-provisioning every resource is the rational design. That reframing matters because it changes what “scarce” means. Compute is not scarce in the way the discourse treats it. What is scarce is a coherent system designed end-to-end. The xAI 11 percent number, in that frame, is not embarrassing. It is the natural reading of a workload that is mostly decode.

    The Stanford segment is the part most likely to be quoted out of context. “It’s Stanford’s fault” is a deliberately provocative line, but the underlying claim is correct and load-bearing. Compute is not gated by NVIDIA refusing to ship chips. It is gated by the fact that fragmented grant funding cannot aggregate into the billion-dollar order that NVIDIA can fulfill. The implication is that universities and national labs need a structural change in how they pool capital for compute, and that the current model of every researcher buying a handful of cards is genuinely obsolete. Huang’s nudge about pulling a billion off the endowment is concrete enough to be acted on, and other major research universities should read this segment as a direct prompt.

    The geopolitical segment is the highest-stakes one. The telecommunications comparison is correct as a historical pattern, and Huang is one of the very few executives in a position to deliver that warning credibly. The unresolved tension is that the argument applies symmetrically. If American AI dominance is built by selling globally, that includes selling into adversarial states, and the policy question is where the line falls. Huang does not answer that question. He attacks the framing that lets the question be answered badly. That is a meaningful contribution to the discourse even if it does not resolve the underlying tradeoff.

    The career advice section is the part the social-media clips will mishandle. “Seek suffering” reads as macho when extracted. In context it is a specific operational claim about how resilience compounds, and it is paired with the Tegra story where Huang himself paid the price of not thinking one more click ahead. That kind of self-implication is rare in CEO talks, and it is the reason the talk is worth listening to in full rather than only reading the recap.

    Watch the full Stanford CS153 Frontier Systems conversation with Jensen Huang here.

  • Andrej Karpathy on the Decade of AI Agents: Insights from His Dwarkesh Podcast Interview

    TL;DR

    Andrej Karpathy’s reflections on artificial intelligence trace the quiet, inevitable evolution of deep learning systems into general-purpose intelligence. He emphasizes that the current breakthroughs are not sudden revolutions but the result of decades of scaling simple ideas — neural networks trained with enormous data and compute resources. The essay captures how this scaling leads to emergent behaviors, transforming AI from specialized tools into flexible learning systems capable of handling diverse real-world tasks.

    Summary

    Karpathy explores the evolution of AI from early, limited systems into powerful general learners. He frames deep learning as a continuation of a natural process — optimization through scale and feedback — rather than a mysterious or handcrafted leap forward. Small, modular algorithms like backpropagation and gradient descent, when scaled with modern hardware and vast datasets, have produced behaviors that resemble human-like reasoning, perception, and creativity.

    He argues that this progress is driven by three reinforcing trends: increased compute power (especially GPUs and distributed training), exponentially larger datasets, and the willingness to scale neural networks far beyond human intuition. These factors combine to produce models that are not just better at pattern recognition but are capable of flexible generalization, learning to write code, generate art, and reason about the physical world.

    Drawing from his experience at OpenAI and Tesla, Karpathy illustrates how the same fundamental architectures power both self-driving cars and large language models. Both systems rely on pattern recognition, prediction, and feedback loops — one for navigating roads, the other for navigating language. The essay connects theory to practice, showing that general-purpose learning is not confined to labs but already shapes daily technologies.

    Ultimately, Karpathy presents AI as an emergent phenomenon born from scale, not human ingenuity alone. Just as evolution discovered intelligence through countless iterations, AI is discovering intelligence through optimization — guided not by handcrafted rules but by data and feedback.

    Key Takeaways

    • AI progress is exponential: Breakthroughs that seem sudden are the cumulative effect of scaling and compounding improvements.
    • Simple algorithms, massive impact: The underlying principles — gradient descent, backpropagation, and attention — are simple but immensely powerful when scaled.
    • Scale is the engine of intelligence: Data, compute, and model size form a triad that drives emergent capabilities.
    • Generalization emerges from scale: Once models reach sufficient size and data exposure, they begin to generalize across modalities and tasks.
    • Parallel to evolution: Intelligence, whether biological or artificial, arises from iterative optimization processes — not design.
    • Unified learning systems: The same architectures can drive perception, language, planning, and control.
    • AI as a natural progression: What humanity is witnessing is not an anomaly but a continuation of the evolution of intelligence through computation.

    Discussion

    The essay invites a profound reflection on the nature of intelligence itself. Karpathy’s framing challenges the idea that AI development is primarily an act of invention. Instead, he suggests that intelligence is an attractor state — something the universe converges toward given the right conditions: energy, computation, and feedback. This idea reframes AI not as an artificial construct but as a natural phenomenon, emerging wherever optimization processes are powerful enough.

    This perspective has deep implications. It implies that the future of AI is not dependent on individual breakthroughs or genius inventors but on the continuation of scaling trends — more data, more compute, more refinement. The question becomes not whether AI will reach human-level intelligence, but when and how we’ll integrate it into our societies.

    Karpathy’s view also bridges philosophy and engineering. By comparing machine learning to evolution, he removes the mystique from intelligence, positioning it as an emergent property of systems that self-optimize. In doing so, he challenges traditional notions of creativity, consciousness, and design — raising questions about whether human intelligence is just another instance of the same underlying principle.

    For engineers and technologists, his message is empowering: the path forward lies not in reinventing the wheel but in scaling what already works. For ethicists and policymakers, it’s a reminder that these systems are not controllable in the traditional sense — their capabilities unfold with scale, often unpredictably. And for society as a whole, it’s a call to prepare for a world where intelligence is no longer scarce but abundant, embedded in every tool and interaction.

    Karpathy’s work continues to resonate because it captures the duality of the AI moment: the awe of creation and the humility of discovery. His argument that “intelligence is what happens when you scale learning” provides both a technical roadmap and a philosophical anchor for understanding the transformations now underway.

    In short, AI isn’t just learning from us — it’s showing us what learning itself really is.

  • AI Breakthrough: Large Language Model GPT-4.5 Passes the Turing Test, Often Appearing ‘More Human’ Than Humans

    In a landmark study suggesting a significant leap in artificial intelligence capabilities, researchers from the University of California San Diego have provided the first robust empirical evidence that a large language model (LLM) can pass the standard Turing test. The study found that OpenAI’s GPT-4.5 model, when guided by a specific prompt to adopt a humanlike persona, was judged to be the human conversational partner 73% of the time, significantly more often than actual human participants were selected.

    The Turing test, proposed by Alan Turing in 1950 as the “imitation game,” is a benchmark for machine intelligence. In its classic three-party form, a human interrogator holds simultaneous text-based conversations with two unseen witnesses – one human, one machine. Both witnesses try to convince the interrogator they are human. If the interrogator cannot reliably distinguish the machine from the human, the machine is said to have passed the test.

    Despite numerous attempts over 75 years, no AI system had previously passed a standard, controlled three-party Turing test. While modern AI benchmarks often focus on narrow, static tasks, the Turing test provides a dynamic, interactive, and adversarial evaluation of an AI’s ability to mimic general human conversational behavior and potentially deceive.

    Researchers Cameron R. Jones and Benjamin K. Bergen conducted two pre-registered, randomized, controlled experiments using independent populations (UCSD undergraduates and online Prolific workers). Participants engaged in 5-minute conversations, acting as the interrogator comparing an AI witness against a human witness. Four AI systems were evaluated: the classic 1960s chatbot ELIZA, OpenAI’s GPT-4o (an earlier model), Meta’s LLaMa-3.1-405B, and OpenAI’s GPT-4.5.

    Crucially, the study tested the impact of prompting. In a baseline “NO-PERSONA” condition, LLMs received minimal instructions. In the key “PERSONA” condition, LLMs (GPT-4.5 and LLaMa-3.1) were prompted to act like a specific type of young, introverted person familiar with internet culture.

    The results were striking:

    1. GPT-4.5 (PERSONA): Achieved a 73% “win rate,” meaning interrogators identified it as human 73% of the time. This was significantly above the 50% chance level and notably higher than the rate at which actual human participants were identified.
    2. LLaMa-3.1 (PERSONA): Achieved a 56% win rate. This was statistically indistinguishable from chance (and thus indistinguishable from the human win rate), meeting the criteria for passing the test.
    3. Baseline Models (ELIZA, GPT-4o NO-PERSONA): Failed significantly, with win rates of only 23% and 21% respectively. This confirmed that participants were not simply guessing randomly or easily fooled by any AI.
    4. NO-PERSONA LLMs (GPT-4.5, LLaMa-3.1): Performed much worse than their PERSONA counterparts (36% and 38% win rates overall), failing to robustly pass the test across both study populations. This highlights the critical role of prompting in achieving humanlike imitation.

    The researchers noted that interrogators often focused more on linguistic style, social, and emotional cues (like tone, humor, or personality) rather than purely factual knowledge or logical reasoning when making their judgments. Interestingly, sometimes demonstrating a lack of knowledge contributed to an AI seeming more human.

    These findings indicate that current leading LLMs, when appropriately prompted, can successfully imitate human conversational partners in short interactions to the point of indistinguishability, and even appear more convincing than actual humans. The authors argue this demonstrates a high degree of “humanlikeness” rather than necessarily proving abstract intelligence in the way Turing originally envisioned.

    The study carries significant social and economic implications. The ability of AI to convincingly pass as human raises concerns about “counterfeit people” online, facilitating social engineering, spreading misinformation, or replacing humans in roles requiring brief conversational interactions. While the test was limited to 5 minutes, the results signal a new era where distinguishing human from machine in online text interactions has become substantially more difficult. The researchers suggest future work could explore longer test durations and different participant populations or incentives to further probe the boundaries of AI imitation.

  • Michael Dell on Building a Tech Empire and Embracing Innovation: Insights from “In Good Company”

    In the December 11, 2024 episode of “In Good Company,” hosted by Nicolai Tangen of Norges Bank Investment Management, Michael Dell, the visionary founder and CEO of Dell Technologies, offers an intimate glimpse into his remarkable career and the strategic decisions that have shaped one of the world’s leading technology companies. This interview not only chronicles Dell’s entrepreneurial journey but also provides profound insights into leadership, innovation, and the future of technology.

    From Bedroom Enthusiast to Tech Titan

    Michael Dell’s fascination with computers began in his teenage years. At 16, instead of using his IBM PC conventionally, he chose to dismantle it to understand its inner workings. This hands-on curiosity led him to explore microprocessors, memory chips, and other hardware components. Dell discovered that IBM’s pricing was exorbitant—charging roughly six times the cost of the parts—sparking his determination to offer better value to customers through a more efficient business model.

    Balancing his academic pursuits at the University of Texas, where he was initially a biology major, Dell engaged in various entrepreneurial activities. From working in a Chinese restaurant to trading stocks and selling newspapers, these early ventures provided him with the capital and business acumen to invest in his burgeoning interest in technology. Despite familial pressures to follow a medical career, Dell’s passion for computers prevailed, leading him to fully commit to his business aspirations.

    The Birth and Explosive Growth of Dell Technologies

    In May 1984, Dell Computer Corporation was officially incorporated. The company experienced meteoric growth, with revenues skyrocketing from $6 million in its first year to $33 million in the second. This impressive 80% annual growth rate continued for eight years, followed by a sustained 60% growth for six more years. Dell’s success was largely driven by his innovative direct-to-consumer sales model, which eliminated intermediaries like retail stores. This approach not only reduced costs but also provided Dell with real-time insights into customer demand, allowing for precise inventory management and rapid scaling.

    Dell attributes this entrepreneurial mindset to curiosity and a relentless pursuit of better performance and value. He believes that America’s culture of embracing risk, supported by accessible capital and inspirational role models like Bill Gates and Steve Jobs, fosters a robust environment for entrepreneurs.

    Revolutionizing Supply Chains and Strategic Business Moves

    A cornerstone of Dell’s strategy was revolutionizing the supply chain through direct sales. This model allowed the company to respond swiftly to customer demands, minimizing inventory costs and enhancing capital efficiency. By maintaining close relationships with a diverse customer base—including individual consumers, large enterprises, and governments—Dell ensured high demand fidelity, enabling the company to scale efficiently.

    In 2013, facing declining stock prices and skepticism about the relevance of PCs amid the rise of smartphones and tablets, Dell made the bold decision to take the company private. This move involved a massive $67 billion buyback of shares, the largest technology acquisition at the time. Going private allowed Dell to focus on long-term transformation without the pressures of quarterly earnings reports.

    The acquisition of EMC, a major player in data storage and cloud computing, was a landmark deal that significantly expanded Dell’s capabilities. Despite initial uncertainties and challenges, the merger proved successful, resulting in substantial organic revenue growth and enhanced offerings for enterprise customers. Dell credits this acquisition for accelerating the company’s transformation and broadening its technological expertise.

    Leadership Philosophy: “Play Nice but Win”

    Dell’s leadership philosophy is encapsulated in his motto, “Play Nice but Win.” This principle emphasizes ethical behavior, fairness, and a strong results orientation. He fosters a culture of open debate and diverse perspectives, believing that surrounding oneself with intelligent individuals who can challenge ideas leads to better decision-making. Dell encourages his team to engage in rigorous discussions, ensuring that decisions are well-informed and adaptable to changing circumstances.

    He advises against being the smartest person in the room, advocating instead for inviting smarter people or finding environments that foster continuous learning and adaptation. This approach not only drives innovation but also ensures that Dell Technologies remains agile and forward-thinking.

    Embracing the Future: AI and Technological Innovation

    Discussing the future of technology, Dell highlights the transformative impact of artificial intelligence (AI) and large language models. He views current AI advancements as the initial phase of a significant technological revolution, predicting substantial improvements and widespread adoption over the next few years. Dell envisions AI enhancing productivity and enabling businesses to reimagine their processes, ultimately driving human progress.

    He also touches upon the evolving landscape of personal computing. While the physical appearance of PCs may not change drastically, their capabilities are significantly enhanced through AI integration. Innovations such as neural processing units (NPUs) are making PCs more intelligent and efficient, ensuring continued demand for new devices.

    Beyond Dell Technologies: MSD Capital and Investment Ventures

    Beyond his role at Dell Technologies, Michael Dell oversees MSD Capital, an investment firm that has grown into a prominent investment boutique on Wall Street. Initially established to manage investments for his family and foundation, MSD Capital has expanded through mergers and strategic partnerships, including a significant merger with BDT. Dell remains actively involved in guiding the firm’s strategic direction, leveraging his business acumen to provide aligned investment solutions for multiple families and clients.

    Balancing Success with Personal Well-being

    Despite his demanding roles, Dell emphasizes the importance of maintaining a balanced lifestyle. He adheres to a disciplined daily routine that includes early waking hours, regular exercise, and sufficient sleep. Dell advocates for a balanced approach to work and relaxation to sustain long-term productivity and well-being. He also underscores the role of humor in the workplace, believing that the ability to laugh and joke around fosters a positive and creative work environment.

    Advice to Aspiring Entrepreneurs

    Addressing the younger audience, Dell offers invaluable advice to aspiring entrepreneurs: experiment, take risks, and embrace failure as part of the learning process. He encourages tackling challenging problems, creating value, and being bold in endeavors. While acknowledging the value of parental guidance, Dell emphasizes the importance of forging one’s own path to achieve success, highlighting that innovation often requires stepping outside conventional expectations.

    Wrap Up

    Michael Dell’s conversation on “In Good Company” provides a deep dive into the strategic decisions, leadership philosophies, and forward-thinking approaches that have propelled Dell Technologies to its current stature. His insights into entrepreneurship, innovation, and the future of technology offer valuable lessons for business leaders and aspiring entrepreneurs alike. Dell’s unwavering commitment to understanding customer needs, fostering a culture of open debate, and leveraging technological advancements underscores his enduring influence in the technology sector.

  • Leveraging Efficiency: The Promise of Compact Language Models

    Leveraging Efficiency: The Promise of Compact Language Models

    In the world of artificial intelligence chatbots, the common mantra is “the bigger, the better.”

    Large language models such as ChatGPT and Bard, renowned for generating authentic, interactive text, progressively enhance their capabilities as they ingest more data. Daily, online pundits illustrate how recent developments – an app for article summaries, AI-driven podcasts, or a specialized model proficient in professional basketball questions – stand to revolutionize our world.

    However, developing such advanced AI demands a level of computational prowess only a handful of companies, including Google, Meta, OpenAI, and Microsoft, can provide. This prompts concern that these tech giants could potentially monopolize control over this potent technology.

    Further, larger language models present the challenge of transparency. Often termed “black boxes” even by their creators, these systems are complicated to decipher. This lack of clarity combined with the fear of misalignment between AI’s objectives and our own needs, casts a shadow over the “bigger is better” notion, underscoring it as not just obscure but exclusive.

    In response to this situation, a group of burgeoning academics from the natural language processing domain of AI – responsible for linguistic comprehension – initiated a challenge in January to reassess this trend. The challenge urged teams to construct effective language models utilizing data sets that are less than one-ten-thousandth of the size employed by the top-tier large language models. This mini-model endeavor, aptly named the BabyLM Challenge, aims to generate a system nearly as competent as its large-scale counterparts but significantly smaller, more user-friendly, and better synchronized with human interaction.

    Aaron Mueller, a computer scientist at Johns Hopkins University and one of BabyLM’s organizers, emphasized, “We’re encouraging people to prioritize efficiency and build systems that can be utilized by a broader audience.”

    Alex Warstadt, another organizer and computer scientist at ETH Zurich, expressed that the challenge redirects attention towards human language learning, instead of just focusing on model size.

    Large language models are neural networks designed to predict the upcoming word in a given sentence or phrase. Trained on an extensive corpus of words collected from transcripts, websites, novels, and newspapers, they make educated guesses and self-correct based on their proximity to the correct answer.

    The constant repetition of this process enables the model to create networks of word relationships. Generally, the larger the training dataset, the better the model performs, as every phrase provides the model with context, resulting in a more intricate understanding of each word’s implications. To illustrate, OpenAI’s GPT-3, launched in 2020, was trained on 200 billion words, while DeepMind’s Chinchilla, released in 2022, was trained on a staggering trillion words.

    Ethan Wilcox, a linguist at ETH Zurich, proposed a thought-provoking question: Could these AI language models aid our understanding of human language acquisition?

    Traditional theories, like Noam Chomsky’s influential nativism, argue that humans acquire language quickly and effectively due to an inherent comprehension of linguistic rules. However, language models also learn quickly, seemingly without this innate understanding, suggesting that these established theories may need to be reevaluated.

    Wilcox admits, though, that language models and humans learn in fundamentally different ways. Humans are socially engaged beings with tactile experiences, exposed to various spoken words and syntaxes not typically found in written form. This difference means that a computer trained on a myriad of written words can only offer limited insights into our own linguistic abilities.

    However, if a language model were trained only on the vocabulary a young human encounters, it might interact with language in a way that could shed light on our own cognitive abilities.

    With this in mind, Wilcox, Mueller, Warstadt, and a team of colleagues launched the BabyLM Challenge, aiming to inch language models towards a more human-like understanding. They invited teams to train models on roughly the same amount of words a 13-year-old human encounters – around 100 million. These models would be evaluated on their ability to generate and grasp language nuances.

    Eva Portelance, a linguist at McGill University, views the challenge as a pivot from the escalating race for bigger language models towards more accessible, intuitive AI.

    Large industry labs have also acknowledged the potential of this approach. Sam Altman, the CEO of OpenAI, recently stated that simply increasing the size of language models wouldn’t yield the same level of progress seen in recent years. Tech giants like Google and Meta have also been researching more efficient language models, taking cues from human cognitive structures. After all, a model that can generate meaningful language with less training data could potentially scale up too.

    Despite the commercial potential of a successful BabyLM, the challenge’s organizers emphasize that their goals are primarily academic. And instead of a monetary prize, the reward lies in the intellectual accomplishment. As Wilcox puts it, the prize is “Just pride.”

  • AI Industry Pioneers Advocate for Consideration of Potential Challenges Amid Rapid Technological Progress

    AI Industry Pioneers Advocate for Consideration of Potential Challenges Amid Rapid Technological Progress

    On Tuesday, a collective of industry frontrunners plans to express their concern about the potential implications of artificial intelligence technology, which they have a hand in developing. They suggest that it could potentially pose significant challenges to society, paralleling the severity of pandemics and nuclear conflicts.

    The anticipated statement from the Center for AI Safety, a nonprofit organization, will call for a global focus on minimizing potential challenges from AI. This aligns it with other significant societal issues, such as pandemics and nuclear war. Over 350 AI executives, researchers, and engineers have signed this open letter.

    Signatories include chief executives from leading AI companies such as OpenAI’s Sam Altman, Google DeepMind’s Demis Hassabis, and Anthropic’s Dario Amodei.

    In addition, Geoffrey Hinton and Yoshua Bengio, two Turing Award-winning researchers for their pioneering work on neural networks, have signed the statement, along with other esteemed researchers. Yann LeCun, the third Turing Award winner, who leads Meta’s AI research efforts, had not signed as of Tuesday.

    This statement arrives amidst escalating debates regarding the potential consequences of artificial intelligence. Innovations in large language models, as employed by ChatGPT and other chatbots, have sparked concerns about the misuse of AI in spreading misinformation or possibly disrupting numerous white-collar jobs.

    While the specifics are not always elaborated, some in the field argue that unmitigated AI developments could lead to societal-scale disruptions in the not-so-distant future.

    Interestingly, these concerns are echoed by many industry leaders, placing them in the unique position of suggesting tighter regulations on the very technology they are working to develop and advance.

    In an attempt to address these concerns, Altman, Hassabis, and Amodei recently engaged in a conversation with President Biden and Vice President Kamala Harris on the topic of AI regulation. Following this meeting, Altman emphasized the importance of government intervention to mitigate the potential challenges posed by advanced AI systems.

    In an interview, Dan Hendrycks, executive director of the Center for AI Safety, suggested that the open letter represented a public acknowledgment from some industry figures who previously only privately expressed their concerns about potential risks associated with AI technology development.

    While some critics argue that current AI technology is too nascent to pose a significant threat, others contend that the rapid progress of AI has already exceeded human performance in some areas. These proponents believe that the emergence of “artificial general intelligence,” or AGI, an AI capable of performing a wide variety of tasks at or beyond human-level performance, may not be too far off.

    In a recent blog post, Altman, along with two other OpenAI executives, proposed several strategies to manage powerful AI systems responsibly. They proposed increased cooperation among AI developers, further technical research into large language models, and the establishment of an international AI safety organization akin to the International Atomic Energy Agency.

    Furthermore, Altman has endorsed regulations requiring the developers of advanced AI models to obtain a government-issued license.

    Earlier this year, over 1,000 technologists and researchers signed another open letter advocating for a six-month halt on the development of the largest AI models. They cited fears about an unregulated rush to develop increasingly powerful digital minds.

    The new statement from the Center for AI Safety is brief, aiming to unite AI experts who share general concerns about powerful AI systems, regardless of their views on specific risks or prevention strategies.

    Geoffrey Hinton, a high-profile AI expert, recently left his position at Google to openly discuss potential AI implications. The statement has since been circulated and signed by some employees at major AI labs.

    The recent increased use of AI chatbots for entertainment, companionship, and productivity, combined with the rapid advancements in the underlying technology, has amplified the urgency of addressing these concerns.

    Altman emphasized this urgency in his Senate subcommittee testimony, saying, “We want to work with the government to prevent [potential challenges].”

  • Revolutionize Your Note-Taking with AI

    Revolutionize Your Note-Taking with AI

    As technology continues to advance, it’s becoming increasingly clear that artificial intelligence (AI) will play a significant role in our lives. In fact, there are some tasks that AI may eventually be able to do better than humans. One such task is organizing notes.

    Many of us have struggled with the task of organizing our notes at one time or another. We create elaborate systems of tags, hierarchies, and links in an effort to make sure we can find the right notes at the right time. However, these systems can be brittle and often fail to deliver the desired results. We may build and abandon new systems frequently, and it’s rare that we go back to look at old notes. This can be frustrating, especially considering the value that is often locked up in the notes we’ve collected over the years.

    AI could potentially solve this problem by using natural language processing to understand the content of our notes and surface relevant ones based on the task at hand. This would make it much easier to find and understand old notes, as the AI would be able to provide context and relevance.

    But why is it so hard to organize notes in the first place? One reason is that it’s difficult to know how to categorize a piece of information when it could potentially be useful for many different purposes. For example, you might write down a quote from a book because you could eventually use it in a variety of ways – to make a decision, to write an essay, or to lift a friend’s spirits. Similarly, notes from a meeting or thoughts about a new person you’ve met could have numerous potential uses.

    Another reason organizing notes is challenging is that it can be cognitively taxing to try to understand old notes and determine their relevance. When you read an old note, you often have to try to recreate the context in which it was written and understand why it was written in the first place. This can be a time-consuming and often unrewarding task. For an old note to be truly helpful, it needs to be presented in a way that makes it easy to understand and use.

    This is where AI comes in. By using natural language processing to understand the content of our notes, an AI system could present old notes in a more digestible format. It could also surface relevant notes based on the task at hand, making it easier to find and use the information we need.

    Of course, there are some limitations to what AI can do. It may not be able to fully understand the nuances and subtleties of human thought and expression. However, as AI continues to improve and advance, it’s possible that it will eventually be able to take over the task of organizing notes for us.

    In the future, large language models like GPT-3 could potentially turn our notes into an “actual” second brain, taking over the task of organization and making it easier for us to find and use the information we need. This could be a game-changer for those of us who have struggled with the task of organizing our notes in the past.