New work to appear @ TACL! Language models (LMs) are remarkably good at generating novel well-formed sentences, leading to claims that they have mastered grammar. Yet they often assign higher probability to ungrammatical strings than to grammatical strings. How can both things be true? 🧵👇
Roger Levy
@rplevy.bsky.social
Director, MIT Computational Psycholinguistics Lab. President, Cognitive Science Society. Chair of the MIT Faculty. Open access & open science advocate. He. Lab webpage: http://cpl.mit.edu/ Personal webpage: https://www.mit.edu/~rplevy
thinking of calling this "The Illusion Illusion" (more examples below)
Been listening a lot to Ella Jenkins the last couple of days. What wonderful music and performance!
Within the past two weeks I deleted apps for media companies owned by 67% of the world’s richest billionaires, and it felt great!
Results of high stakes elections that happen only once every four years offer remarkable opportunities for overfitting theories of the electorate
It's my book's release day! The Patterns of Comics is now officially published, featuring an extended data-driven analysis of the structures used in 350+ comics from Asia, Europe, and North America analyzing diversity, regularity, and change over time www.visuallanguagelab.com/poc
GOOD MORNING BLUESKY! Very excited about this new paper: www.pnas.org/doi/10.1073/pnas.2300671120 Key Q: what predicts how much young kids (👶)talk? How much 🗣 kids heard predicted how much 👶talked, but other factors, e.g. mom’s education, didn’t. #PsychSci #DevPsy 🗣💬 INCOMING SUMMARY🧵ALERT 1/14
#linguistics Bluesky: what are the best available quantitative measures of dialect/language mutual intelligibility? The more fine-grained, the better: I'm hoping to vividly illustrate at least one specific dialect continuum (e.g., the Romance languages of the Mediterranean coast)
Today in linguists are NOT KIDDING when we say that your capacity for language enables you to understand sentences that have never before been uttered in human history.
New postdoc opportunity to work jointly with @cantlonlab.bsky.social and me to understand cognition across species, age, and culture! cmu.wd5.myworkdayjobs.com/CMU/job/Pitt...
Absolutely, the Nature EiC has it completely backwards. Checking for errors and quality of data (and of math, code, and argumentation) is the most important work that reviewers can do.
The quotes from Nature EiC Magdalena Skipper about whether journals should be checking for errors/data quality as part of peer review are quite surprising to me. https://www.wsj.com/science/whats-wrong-with-peer-review-e5d2d428?st=dhrnljoa74fujcv&reflink=desktopwebshare_permalink
The quotes from Nature EiC Magdalena Skipper about whether journals should be checking for errors/data quality as part of peer review are quite surprising to me. https://www.wsj.com/science/whats-wrong-with-peer-review-e5d2d428?st=dhrnljoa74fujcv&reflink=desktopwebshare_permalink
I am delighted to announce that the Department of Biology at the University of Washington is advertising for a tenure-track assistant professor position on the quantitative understanding of collective behavior. I will be chairing the search; details are here: apply.interfolio.com/130336
I think it’s amazing that Cognitive Science gives recent PhDs $10K in UNRESTRICTED CASH…right when folks are broke, exhausted, moving town…and need it most. It’s almost Glushko season! cognitivesciencesociety.org/glushko-diss...
Glushko Dissertation Prize - Cognitive Science Society
cognitivesciencesociety.org
First post and big news - I am starting as an Assistant Professor in Psychology at Georgia Tech in Jan 2024! www.language-intelligence-thought.net
LIT Lab
Our goal is to understand the relationship between language and human thought. How does the language network in the brain interact with other systems to interpret meaning in the world? Can models...
language-intelligence-thought.net
In a new TiCS article, @emaliemcmahon.bsky.social and I review a growing body of behavioral, neural, and computational evidence that social interactions are automatically extracted by the human visual system: tinyurl.com/nhh2dhxt #PsychSciSky #NeuroSkyence
While the world has its eyes on the Middle East, democratic conditions in Indonesia are looking grim. The Supreme Court has overruled the Constitution in order to allow the sitting president's son to stand as Vice Presidential candidate with a disgraced general with a stained human rights record
I am reading PhD applications this year, with a special interest in students who would like to work on the topic of perceived danger. But open to all applicants who share some of my interests. Visit www.liulaboratory.org to see papers, lab values, and tips for application writing.
LIU LAB
In the Look, Infer, and Understand (LIU) Lab at Johns Hopkins University, we are interested in how our minds and brains reason about the physical and social world. We study the developmental and neu...
liulaboratory.org
The Consumer Financial Protection Bureau (CFPB) is hiring a section chief for the psych/”behavioral” section and I'd love to see some CogSci representation in there! My brother works there (trained as an economist) and it's an incredible gig doing research in the public interest.
Supervisory Research Scientist (Interdisciplinary)
This position is located at the Consumer Financial Protection Bureau (CFPB), Office of Research. The incumbent supervises and conducts independent, self-directed social/behavioral analysis on a variet...
usajobs.gov
Santa Fe Institute now has a Blue Sky account: @sfiscience.bsky.social
A new cross-linguistic study on demonstratives by a team of psychologists and linguists: "Commonalities and differences across languages in spatial communication can be understood in terms of universal constraints on action shaping spatial language and cognition." www.nature.com/articles/s41...
Spatial communication systems across languages reflect universal action constraints - Nature Human B...
Coventry et al. show that spatial demonstratives—such as ‘this’ and ‘that’ in English—are selected on the basis of whether the speaker is able to reach the object or not, across 29 diverse...
nature.com
New paper out! "Large language models show human-like content biases in transmission chain experiments" #CulturalEvolution #cssky 🧪 www.pnas.org/doi/10.1073/...
PNAS
Proceedings of the National Academy of Sciences (PNAS), a peer reviewed journal of the National Academy of Sciences (NAS) - an authoritative source of high-impact, original research that broadly spans...
pnas.org
new work just dropped, see @stephan-meylan.bsky.social's "thread" below:) #DevPsych #CogPsych #PsychSciSky #CogSciSky
How do adults understand children’s early, highly variable speech? Our new paper in Nature Human Behavior (www.nature.com/articles/s41...) provides evidence that adults’ interpretations depend quite strongly on language expectations—what they think children are likely to say. 1/
Thrilled at publication of @stephan-meylan.bsky.social's "How adults understand what young children say", featuring Bayesian noisy-channel inference, LLMs, & child speech datasets! TL;DR: prior expectations of what kids *want to say* is crucial. (Knowing how kids mispronounce words is too.)
How do adults understand children’s early, highly variable speech? Our new paper in Nature Human Behavior (www.nature.com/articles/s41...) provides evidence that adults’ interpretations depend quite strongly on language expectations—what they think children are likely to say. 1/
Prompting is *not a substitute* for probability measurements in large language models – the amazing @jennhu.bsky.social's EMNLP paper now available camera-ready! TL;DR: direct probability measurements show LLMs' linguistic generalizations are better than suggested by prompt-based tests.
To researchers doing LLM evaluation: prompting is *not a substitute* for direct probability measurements. Check out the camera-ready version of our work, to appear at EMNLP 2023! (w/ @rplevy.bsky.social) Paper: arxiv.org/abs/2305.13264 Original thread: twitter.com/_jennhu/stat...
Each week I'm posting the first page of a chapter of my book. Here's the first page of chapter 5, "Language and Land" (seven weeks to publication of "The Unnaming of Kroeber Hall: Language, Memory, and Indigenous California", mitpress.mit.edu/9780262547093/).
I was going to submit a manuscript to eLife this week. Just signed this open letter and wrote to my co-authors about submitting our paper somewhere else
Mostly not here for politics. But, at the science/politics intersection: @elife.bsky.social Editor-in-Chief Michael Eisen is apparently being ousted for tweet-referencing a satire conveying the tragedy of the Israeli–Palestinian conflict. Concerned about this form of censure? Sign this open letter.
This is a very precisely written letter, and as a result, I think, a very strong letter, in support of Eisen — I'm happy to be able to sign on. Please consider adding your support, even if anonymously.
Mostly not here for politics. But, at the science/politics intersection: @elife.bsky.social Editor-in-Chief Michael Eisen is apparently being ousted for tweet-referencing a satire conveying the tragedy of the Israeli–Palestinian conflict. Concerned about this form of censure? Sign this open letter.
There already seems to have been one resignation from the editorial board over this:
Tweet by @LaraUrban42
twitter.com