Keyon Vafa

@keyonv.bsky.social

Postdoctoral fellow at Harvard Data Science Initiative | Former computer science PhD at Columbia University | ML + NLP + social sciences https://keyonvafa.com

Can an AI model predict perfectly and still have a terrible world model? What would that even mean? Our new ICML paper (poster tomorrow!) formalizes these questions. One result tells the story: A transformer trained on 10M solar systems nails planetary orbits. But it botches gravitational laws 🧵

If we know someone’s career history, how well can we predict which jobs they’ll have next? Read our profile of @keyonv.bsky.social to learn how ML models can be used to predict workers’ career trajectories & better understand labor markets. medium.com/@gsb_silab/k...

Keyon Vafa: Predicting Workers’ Career Trajectories to Better Understand Labor Markets

If we know someone’s career history, how well can we predict which job they’ll have next?

medium.com

At NeurIPS today through Sunday! Today I'll be presenting our spotlight paper on evaluating LLM world models at the 4:30pm poster session (#2301). On Saturday I'll be co-organizing the Behavioral ML workshop. Hope to see you there! Paper: arxiv.org/abs/2406.03689 Workshop: behavioralml.org

Evaluating the World Model Implicit in a Generative Model

Recent work suggests that large language models may implicitly learn world models. How should we assess this possibility? We formalize this question for the case where the underlying reality is govern...

arxiv.org