Roger Levy

@rplevy.bsky.social

Director, MIT Computational Psycholinguistics Lab. President, Cognitive Science Society. Chair of the MIT Faculty. Open access & open science advocate. He. Lab webpage: http://cpl.mit.edu/ Personal webpage: https://www.mit.edu/~rplevy

New work to appear @ TACL! Language models (LMs) are remarkably good at generating novel well-formed sentences, leading to claims that they have mastered grammar. Yet they often assign higher probability to ungrammatical strings than to grammatical strings. How can both things be true? 🧵👇

Screenshot of a figure with two panels, labeled (a) and (b). The caption reads: "Figure 1: (a) Illustration of messages (left) and strings (right) in toy domain. Blue = grammatical strings. Red = ungrammatical strings. (b) Surprisal (negative log probability) assigned to toy strings by GPT-2."

#linguistics Bluesky: what are the best available quantitative measures of dialect/language mutual intelligibility? The more fine-grained, the better: I'm hoping to vividly illustrate at least one specific dialect continuum (e.g., the Romance languages of the Mediterranean coast)

While the world has its eyes on the Middle East, democratic conditions in Indonesia are looking grim. The Supreme Court has overruled the Constitution in order to allow the sitting president's son to stand as Vice Presidential candidate with a disgraced general with a stained human rights record

I am reading PhD applications this year, with a special interest in students who would like to work on the topic of perceived danger. But open to all applicants who share some of my interests. Visit www.liulaboratory.org to see papers, lab values, and tips for application writing.

LIU LAB

In the Look, Infer, and Understand (LIU) Lab at Johns Hopkins University, we are interested in how our minds and brains reason about the physical and social world. We study the developmental and neu...

liulaboratory.org

The Consumer Financial Protection Bureau (CFPB) is hiring a section chief for the psych/”behavioral” section and I'd love to see some CogSci representation in there! My brother works there (trained as an economist) and it's an incredible gig doing research in the public interest.

Supervisory Research Scientist (Interdisciplinary)

This position is located at the Consumer Financial Protection Bureau (CFPB), Office of Research. The incumbent supervises and conducts independent, self-directed social/behavioral analysis on a variet...

usajobs.gov

A new cross-linguistic study on demonstratives by a team of psychologists and linguists: "Commonalities and differences across languages in spatial communication can be understood in terms of universal constraints on action shaping spatial language and cognition." www.nature.com/articles/s41...

Spatial communication systems across languages reflect universal action constraints - Nature Human B...

Coventry et al. show that spatial demonstratives—such as ‘this’ and ‘that’ in English—are selected on the basis of whether the speaker is able to reach the object or not, across 29 diverse...

nature.com

Thrilled at publication of @stephan-meylan.bsky.social's "How adults understand what young children say", featuring Bayesian noisy-channel inference, LLMs, & child speech datasets! TL;DR: prior expectations of what kids *want to say* is crucial. (Knowing how kids mispronounce words is too.)

Bild
Stephan Meylan@stephan-meylan.bsky.social · 3y ago

How do adults understand children’s early, highly variable speech? Our new paper in Nature Human Behavior (www.nature.com/articles/s41...) provides evidence that adults’ interpretations depend quite strongly on language expectations—what they think children are likely to say. 1/

Figure showing the difference in performance between pour  best model with rich expectations (90% agreement with annotators) and a baseline model that only uses phonetics (42%). We label the difference between the model results as the contribution of language expectations.

Prompting is *not a substitute* for probability measurements in large language models – the amazing @jennhu.bsky.social's EMNLP paper now available camera-ready! TL;DR: direct probability measurements show LLMs' linguistic generalizations are better than suggested by prompt-based tests.

Jennifer Hu@jennhu.bsky.social · 3y ago

To researchers doing LLM evaluation: prompting is *not a substitute* for direct probability measurements. Check out the camera-ready version of our work, to appear at EMNLP 2023! (w/ @rplevy.bsky.social) Paper: arxiv.org/abs/2305.13264 Original thread: twitter.com/_jennhu/stat...

I was going to submit a manuscript to eLife this week. Just signed this open letter and wrote to my co-authors about submitting our paper somewhere else

Roger Levy@rplevy.bsky.social · 3y ago

Mostly not here for politics. But, at the science/politics intersection: @elife.bsky.social Editor-in-Chief Michael Eisen is apparently being ousted for tweet-referencing a satire conveying the tragedy of the Israeli–Palestinian conflict. Concerned about this form of censure? Sign this open letter.

This is a very precisely written letter, and as a result, I think, a very strong letter, in support of Eisen — I'm happy to be able to sign on. Please consider adding your support, even if anonymously.

Roger Levy@rplevy.bsky.social · 3y ago

Mostly not here for politics. But, at the science/politics intersection: @elife.bsky.social Editor-in-Chief Michael Eisen is apparently being ousted for tweet-referencing a satire conveying the tragedy of the Israeli–Palestinian conflict. Concerned about this form of censure? Sign this open letter.