The Bitter Lesson for the Bitter Lesson: Where Human Mathematicians and Engineers go in the Age of AI

This talk argues that the "Bitter Lesson" (Rich Sutton's 2019 claim that general compute-scaling methods dominate) itself needs to learn a lesson. The core thesis: pure ML/AI has fundamental limitations that Scientific Machine Learning (SciML) addresses.Key Problems with Pure ML/AI:Generalization failures - Examples like Tesla autopilot confusing traffic lights with moons, or trucks carrying traffic lightsReward hacking in RL - Agents find absurd exploits (suicidal agents, slamming tables, throwing hammers)Data inefficiency - RL requires ~83 hours of play to match human Atari performance achieved in minutesExponential costs - AlphaGo ($35M), GPT-3 ($10M), AlphaStar (>$1B in research), GPT-5 ($500M training)SciML Solution:The more physics/domain structure encoded into models, the less there is to learn, making algorithms more data-efficient. Universal Differential Equations (UDEs) replace only the unknown portions of physics models with neural networks, enabling accurate extrapolation that pure ML cannot achieve.Applications cited:COVID epidemiology modelingBlack hole gravitational wave analysisEarthquake-resistant building designPumas pharmacometrics (used by Moderna for vaccine development)Agentic AI Section:The talk transitions to practical AI tooling for engineers. Key insight: agentic AI works not because LLMs are accurate (~65% first try), but because the compile/test loop provides ground truth for iteration (~95%+ after loop).Dyad Introduction:JuliaHub's modeling platform designed specifically for agentic AI workflows: simpler code for AI to write/debug, static checks for accuracy, physics-based modeling with generative AI integration. Built on Julia and SciML, targeting aerospace, automotive, and manufacturing.