Speaker: Keyon Vafa

Institution: Harvard Data Science Initiative

Date: 14:00 BST, 27th August, 2025

Talk Title: What are AI’s World Models?

Abstract: Humans can build predictive models and use them to make scientific discoveries; Kepler’s predictions of planetary motion later led to the discovery of Newtonian mechanics. Generative models in AI can make great predictions. But have they made the transition to developing a deeper kind of understanding? Testing for understanding requires first defining it; this talk will propose theoretically-grounded definitions and metrics for assessing world models in AI. We will focus on two settings: one where models are designed to perform a single task, and another where a foundation model is intended to perform many tasks. These exercises demonstrate that models can make highly accurate predictions with incoherent world models, revealing their fragility.

Bio: Keyon Vafa is a postdoctoral fellow at Harvard University, affiliated with the Harvard Data Science Initiative. His research focuses on behavioral machine learning, where he develops tools to evaluate how AI models understand the world with the goal of fostering shared understanding between models and people. He is also an affiliate of the Laboratory for Information and Decision Systems (LIDS) at MIT. Vafa earned his Ph.D. in computer science from Columbia University in 2023, where he was advised by David Blei. During his doctoral studies, he was supported by the NSF Graduate Research Fellowship and the Cheung-Kong Innovation Doctoral Fellowship, and he received the Morton B. Friedman Memorial Prize for excellence in engineering upon graduation.