If I had to summarize this paper in one philosophical question: Why do we call some things by the same name and others by different names? Related to old philosophy of logic/language is Quine's work (1960): en.wikipedia.org/wiki/Inscrut...

Benno Krojer@bennokrojer.bsky.social · 2mo ago

My first last-author paper is out! If you saw this dog below and someone showed you the second image, would you consider them the same word/concept? (more examples in Ada's thread) We study if VLMs agree with humans on this and revisit old questions around shape vs. texture bias in vision

My first last-author paper is out! If you saw this dog below and someone showed you the second image, would you consider them the same word/concept? (more examples in Ada's thread) We study if VLMs agree with humans on this and revisit old questions around shape vs. texture bias in vision

Ada@adadtur.bsky.social · 2mo ago

Super excited to finally announce my latest research “Would you still call this Dax? Novel Visual References in VLMs and Humans”! We studied how vision-language models (VLMs) adopt new visual concepts and map them to language compared to humans, and found that…

I'll be presenting OvertonBench at #ICLR2026 in Rio later this week! 📍Sat, Apr 25, 10:30am in Pavilion 4 (#4109) Please DM me if you'd like to chat about pluralistic / value alignment, societal impacts, epistemology, fairness, evals, etc

Elinor@elinorpd.bsky.social · 5mo ago

There's been a lot of excitement about pluralistic value alignment 🌈 — AI that reflects the full range of human perspectives But no formal way to benchmark whether we're actually making progress. 🤔 Introducing 𝐎𝐕𝐄𝐑𝐓𝐎𝐍𝐁𝐄𝐍𝐂𝐇. 🎉Accepted to #ICLR2026 1/n 🧵

I'll be at ICLR! Will present our LatentLens paper at the Re-Align workshop and happy to chat (ideally at the beach 🏖️)

Benno Krojer@bennokrojer.bsky.social · 6mo ago

🚨New paper Are visual tokens going into an LLM interpretable 🤔 Existing methods (e.g. logit lens) and assumptions would lead you to think “not much”... We propose LatentLens and show that most visual tokens are interpretable across *all* layers 💡 Details 🧵

What are your favorite papers that can serve as excellent examples how to write great scientific paper, present results, great figures, make it engaging and easy to follow? Doesn't necessarily have to be the most cited or impactful ones

Inspired by @bennokrojer.bsky.social, we included a Behind the Scenes section 🎬 The goal is to make science more transparent 🔍, share lessons learned 🧠, and provide a more realistic lens on the research journey 👣 8/ bsky.app/profile/benn...

Benno Krojer@bennokrojer.bsky.social · 6mo ago

For every one of my papers, I try to include a "Behind the Scenes" section I think this paper in particular has a lot going on behind the scenes; from lessons learned to personal reflections let me share some

🚨New Paper!🚨 How do reasoning LLMs handle inferences that have no deterministic answer? We find that they diverge from humans in some significant ways, and fail to reflect human uncertainty… 🧵(1/10)

People often say (myself too): Interpretability on AI is so much easier than neuroscience! We can inspect everything and even retrain (vs carefully poke a little into the brain)! One big advantage in neuroscience I often forget: We're quite literally *inside* the thing we're studying

You can now "pip install latentlens" 🔨 It comes with: * pre-computed embeddings for several popular LLMs and VLMs * a txt file with sentences describing WordNet concepts, which we recommend as a standard corpus to get embeddings from * ... Try it out and let us know what we can improve!

Benno Krojer@bennokrojer.bsky.social · 6mo ago

🚨New paper Are visual tokens going into an LLM interpretable 🤔 Existing methods (e.g. logit lens) and assumptions would lead you to think “not much”... We propose LatentLens and show that most visual tokens are interpretable across *all* layers 💡 Details 🧵

Finally getting into this classic Let's see if by the end I'll have a clearer idea what type of science some fields of AI are, like interpretability What are our paradigms?

Google decided to show this as my first sentence from my website (and not any of the sentences actually at the top of the website)

What does it mean for visual tokens to be "interpretable" to LLM? And how to we measure it? These, and many more pressing questions are addressed! Introducing LatentLens -- a new, more faithful tool for interpretability! Honoured to have collaborated with @bennokrojer.bsky.social on this!

Benno Krojer@bennokrojer.bsky.social · 6mo ago

🚨New paper Are visual tokens going into an LLM interpretable 🤔 Existing methods (e.g. logit lens) and assumptions would lead you to think “not much”... We propose LatentLens and show that most visual tokens are interpretable across *all* layers 💡 Details 🧵

For every one of my papers, I try to include a "Behind the Scenes" section I think this paper in particular has a lot going on behind the scenes; from lessons learned to personal reflections let me share some

Benno Krojer@bennokrojer.bsky.social · 6mo ago

🚨New paper Are visual tokens going into an LLM interpretable 🤔 Existing methods (e.g. logit lens) and assumptions would lead you to think “not much”... We propose LatentLens and show that most visual tokens are interpretable across *all* layers 💡 Details 🧵

This will be my last paper of the phd, can't believe it's been almost 5 years! It is the work i am most proud of and believe has the most potential. Feels right to wrap it up with this one

Benno Krojer@bennokrojer.bsky.social · 6mo ago

🚨New paper Are visual tokens going into an LLM interpretable 🤔 Existing methods (e.g. logit lens) and assumptions would lead you to think “not much”... We propose LatentLens and show that most visual tokens are interpretable across *all* layers 💡 Details 🧵

🚨New paper Are visual tokens going into an LLM interpretable 🤔 Existing methods (e.g. logit lens) and assumptions would lead you to think “not much”... We propose LatentLens and show that most visual tokens are interpretable across *all* layers 💡 Details 🧵

The visual world is composed of objects, and those objects are composed of features. But do VLMs exploit this compositional structure when processing multi-object scenes? In our 🆒🆕 #ICLR2026 paper, we find they do – via emergent symbolic mechanisms for visual binding. 🧵👇

📢 I am hiring a highly-motivated Ph.D student at the University of Copenhagen to work on tokenization-free NLP. Read our previous work in this topic: aclanthology.org/2025.emnlp-m... aclanthology.org/2023.emnlp-m... openreview.net/forum?id=FkS... Apply by March 8: employment.ku.dk/phd/?show=1563

nooo claude's --verbose is gone/broken, how will i now catch wrong assumptions before it goes on for 10 minutes?

A paper that should get more attention, for those interested in building truly multimodal models (aka not just plugging stuff post hoc into LLMs): arxiv.org/abs/2412.06646 TLDR: Counterintuitively, native multimodal modes seem *less* unified internally (interp)

The Narrow Gate: Localized Image-Text Communication in Native Multimodal Models

Recent advances in multimodal training have significantly improved the integration of image understanding and generation within a unified model. This study investigates how vision-language models (VLM...

arxiv.org

Epstein’s economic power among academics was made possible by a capitalist system that makes higher education dependent on the charity economy rather than a public good supported by taxing the rich

🎉 Excited to share our new paper which was accepted to #AAAI2026! As LLMs become increasingly used as sources of factual knowledge, we ask: Do they perform equitably across users of different backgrounds? 🧵⬇️ 1/6

Figuring out the right rules+practices for my CLAUDE.md file and generally how to prompt Claude Code, has taught me more about software/ML engineering paradigms/concepts/best-practices than anything I've learned in uni

Listenining to Michelle Obama's audiobook "Becoming" (loving it btw) in 2026 is wild, an almost comical contrast to now... What hopeful times it was back then, as I'm now at the chapters (2006-2008) describing their 2008 run for president

Wild output from Gemini i got two weeks ago i wonder if the appearance on tv is meant to represent the sin of pride

With the latest coding agents there's almost no excuse anymore to publish a paper without any cool demo or interactive data explorer For my current paper it made it so much easier to quickly grasp different interp tools and their effects throughout the whole project

New blog post 📜 "Better late than never: Getting into interpretability in 2025" It's been a great year pivoting into interp and i wanted to reflect on it bennokrojer.com/interp.html

Really enjoyed the discussions in the UT Austin NLP group!

Juan Diego Rodriguez@juand-r.bsky.social · 9mo ago

Thank you @bennokrojer.bsky.social for presenting your research on #interpretability for VLMs at UT Austin's NLP reading group! I'm looking forward to the preprint. #NLProc #AI

Couldn't have wished for a better place to do my PhD, come apply!

Mila - Institut québécois d'IA@mila-quebec.bsky.social · 10mo ago

Mila's annual supervision request process is now open to receive MSc and PhD applications for Fall 2026 admission! For more information, visit mila.quebec/en/prospecti...

I'll be at COLM! Excited to chat about about anything vision+language, interpretability, cogsci/psych, embedding spaces, visual reasoning, video/world models

Devoured this book in 18 hours, usually not a big fan of audio books! It covered lots from crowdworker rights, the ideologies (doomers, EA, ...) and the silicon valley startup world to the many big egos and company-internal battles Great work by @karenhao.bsky.social