"She'll go to France or Spain or Germany or France" sounds redundant—unless licensed with context like "She will go to an art program in France or Spain or a math program in Germany or France" In new work w/ @qyao.bsky.social and @kmahowald.bsky.social: an LM-based 'Neural Semantics' account 🧵👇

Delighted Sasha's (first year PhD!) work using mech interp to study complex syntax constructions won an Outstanding Paper Award at EMNLP! Also delighted the ACL community continues to recognize unabashedly linguistic topics like filler-gaps... and the huge potential for LMs to inform such topics!

Sasha Boguraev@sashaboguraev.bsky.social · last yr.

A key hypothesis in the history of linguistics is that different constructions share underlying structure. We take advantage of recent advances in mechanistic interpretability to test this hypothesis in Language Models. New work with @kmahowald.bsky.social and @cgpotts.bsky.social! 🧵👇!

UT Austin Linguistics is hiring in computational linguistics! Asst or Assoc. We have a thriving group sites.utexas.edu/compling/ and a long proud history in the space. (For instance, fun fact, Jeff Elman was a UT Austin Linguistics Ph.D.) faculty.utexas.edu/career/170793 🤘

Excited to present this at COLM tomorrow! (Tuesday, 11:00 AM poster session)

Juan Diego Rodriguez@juand-r.bsky.social · last yr.

One of the ways that LLMs can be inconsistent is the "generator-validator gap," where LLMs deem their own answers incorrect. 🎯 We demonstrate that ranking-based discriminator training can significantly reduce this gap, and improvements on one task often generalize to others! 🧵👇

Heading to #COLM2025 to present my first paper w/ @jennhu.bsky.social @kmahowald.bsky.social ! When: Tuesday, 11 AM – 1 PM Where: Poster #75 Happy to chat about my work and topics in computational linguistics & cogsci! Also, I'm on the PhD application journey this cycle! Paper info 👇:

Siyuan Song@siyuansong.bsky.social · last yr.

New preprint w/ @jennhu.bsky.social @kmahowald.bsky.social : Can LLMs introspect about their knowledge of language? Across models and domains, we did not find evidence that LLMs have privileged access to their own predictions. 🧵(1/8)

I’m at #COLM2025 from Wed with: @siyuansong.bsky.social Tue am introspection arxiv.org/abs/2503.07513 @qyao.bsky.social Wed am controlled rearing: arxiv.org/abs/2503.20850 @sashaboguraev.bsky.social INTERPLAY ling interp: arxiv.org/abs/2505.16002 I’ll talk at INTERPLAY too. Come say hi!

Language Models Fail to Introspect About Their Knowledge of Language

There has been recent interest in whether large language models (LLMs) can introspect about their own internal states. Such abilities would make LLMs more interpretable, and also validate the use of s...

arxiv.org

I will be giving a short talk on this work at the COLM Interplay workshop on Friday (also to appear at EMNLP)! Will be in Montreal all week and excited to chat about LM interpretability + its interaction with human cognition and ling theory.

Sasha Boguraev@sashaboguraev.bsky.social · last yr.

A key hypothesis in the history of linguistics is that different constructions share underlying structure. We take advantage of recent advances in mechanistic interpretability to test this hypothesis in Language Models. New work with @kmahowald.bsky.social and @cgpotts.bsky.social! 🧵👇!

The compling group at UT Austin (sites.utexas.edu/compling/) is looking for PhD students! Come join me, @kmahowald.bsky.social, and @jessyjli.bsky.social as we tackle interesting research questions at the intersection of ling, cogsci, and ai! Some topics I am particularly interested in:

LMs learn argument-based preferences for dative constructions (preferring recipient first when it’s shorter), consistent with humans. Is this from memorizing preferences in training? New paper w/ @kanishka.bsky.social , @weissweiler.bsky.social , @kmahowald.bsky.social arxiv.org/abs/2503.20850