Did you know gifting four koi 🐟 is offensive in Japan (4 = "shi" = death 💀), or that chopsticks 🥢upright in rice is inappropriate in China (like funeral incense 🕯️)? Top VLMs don't! Our #COLM2026 paper introduces 🌏NormViz to measure visual norm understanding in 16 countries. Best model: 25.3% 🚨 🧵👇

I am grateful to the ACL executive committee for nominating me for the position of ACL VP Elect.The ballot is now open for ACL members to vote. We are up against unprecedented challenges in the world and in our community, but if we join forces, we can move forward in synergy, strength, and success.

Is the safety-capability tradeoff for LLMs real? Or could it be an artefact of the benchmarks we use to measure safety?? We did some explorations with psychometrics-inspired multi-dimensional IRT models and created BenchMIRT to explore these questions! See 🧵

Ai2@ai2.bsky.social · last mo.

Do LLM safety & capability evals measure what they claim to? We built BenchMIRT to audit them + see which model abilities their Qs actually test. On BBQ, a social-bias eval, it found the Qs distinguished models more by reasoning ability than safety. 🧵 buff.ly/bTcvqJf

I can't believe the first CMU Sapling Xuhui Zhou is now a Doctor! My first solo advisee is now a PhD!! HUGE Congrats @nlpxuhui.bsky.social , I'm so proud, and grateful to have been able to work together!

Speech translation scores well on benchmarks, but is it actually usable when it counts? 🎙️ New paper: Ouvia, a user-centered framework for evaluating speech translation in real-world 1-to-1 communication. 1,700+ interactions, 174 speakers, 3 dialect groups. github.com/g8a9/ouvia #nlp #nlproc

LLMs refuse ambiguous queries that look harmful but aren't. Can they recover once users clarify, while staying safe? Our new interactive multi-turn benchmark measures both. 🚨 Turns out: not both at once.

🚨We are excited to announce the 2nd Pluralistic Alignment Workshop at #ICML2026 in Seoul! Submissions: openreview.net/group?id=ICM... 🗓️Deadline: May 3 More details pluralistic-alignment.github.io We invite work on pluralistic alignment across technical, philosophical, and societal perspectives!

🎨💻 What is a “high-quality” or “aesthetic” image according to generative AI developers? Happy to share that our investigation of the LAION-Aesthetics Predictor has been accepted at #FAccT2026! 🧵 (1/5) Take a look at a preprint here: arxiv.org/abs/2601.09896

Presenting two papers! On Tuesday at 12:40pm (Room I; 12pm session): 1-2-3 Check: Enhancing Contextual Privacy in LLM via Multi-Agent Reasoning arxiv.org/abs/2508.07667 (followed by poster session at 1pm)

1-2-3 Check: Enhancing Contextual Privacy in LLM via Multi-Agent Reasoning

Addressing contextual privacy concerns remains challenging in interactive settings where large language models (LLMs) process information from multiple sources (e.g., summarizing meetings with private...

arxiv.org

Maarten Sap@maartensap.bsky.social · 8mo ago

On my way to Paris for the IASEAI conference www.iaseai.org/our-programs...! Who will be there?

You can read more in the full paper: www.researchgate.net/publication/... There is also an interactive web that contains logs of the authentic interactions: agentsofchaos.baulab.info

(PDF) Agents of Chaos

PDF | We report an exploratory red-teaming study of autonomous language-model-powered agents deployed in a live laboratory environment with persistent... | Find, read and cite all the research you nee...

researchgate.net

In this amazing multidisciplinary collaboration, we report our early experience with the @openclaw-x.bsky.social ->

On my way to Paris for the IASEAI conference www.iaseai.org/our-programs...! Who will be there?

IASEAI - International Association for Safe and Ethical AI

Building a global movement for safe and ethical AI. Join IASEAI to ensure AI systems operate safely and ethically, benefiting all of humanity.

iaseai.org

🚀 Apply to CMU LTI’s Summer 2026 “Language Technology for All” internship! 🎓 Open to pre‑doctoral students new to language tech (non‑CS backgrounds welcome). 🔬 12–14 weeks in‑person in Pittsburgh — travel + stipend paid. 💸 Deadline: Feb 20, 11:59pm ET. Apply → forms.gle/cUu8g6wb27Hs...

CMU LTI Summer 2026 Internship Program Application

We are looking for applicants for the Carnegie Mellon University Language Technology Institute's Summer 2026 "Language Technology for All" internship program. The main goal of this internship is to pr...

forms.gle

I'm excited to announce the Call for Papers for the Social Context (SoCon) and Integrating NLP and Psychology to Study Social Interactions (NLPSI) workshop, @ LREC '26 in Palma de Mallorca, Spain! 🗓Deadline: February 16, 2026 🌐Website: socon-nlpsi.github.io 🗓Workshop: May 12, 2026

SoCon-NLPSI'26 | Home

Natural Language Processing (NLP) has undergone a significant evolution, opening up the possibility of capturing high-level aspects of human communication. Key areas of interest include the pragmatics...

socon-nlpsi.github.io

I'm very excited about our new work which aims to model causes and effects on stories online! Narratives and stories are everywhere, so it's helpful to be able to understand how people use them in nuanced ways.

Joel Mire@joelmire.bsky.social · 10mo ago

Reading social media stories evokes a wide range of contextual reader reactions—inferential, affective, evaluative—yet we lack methods to study these at scale. Excited to share our new paper that builds a framework for analyzing storytelling practices across online communities!

How and when should LLM guardrails be deployed to balance safety and user experience? Our #EMNLP2025 paper reveals that crafting thoughtful refusals rather than detecting intent is the key to human-centered AI safety. 📄 arxiv.org/abs/2506.00195 🧵[1/9]

📣📣 Announcing the first PersonaLLM Workshop on LLM Persona Modeling. If you work on persona driven LLMs, social cognition, HCI, psychology, cognitive science, cultural modeling, or evaluation, do not miss the chance to submit. Submit here: openreview.net/group?id=Neu...

I’m ✨ super excited and grateful ✨to announce that I'm part of the 2025 class of #PackardFellows (www.packard.org/2025fellows). The @packardfdn.bsky.social and this fellowship will allow me to explore exciting research directions towards culturally responsible and safe AI 🌍🌈

🚨New paper: Reward Models (RMs) are used to align LLMs, but can they be steered toward user-specific value/style preferences? With EVALUESTEER, we find even the best RMs we tested exhibit their own value/style biases, and are unable to align with a user >25% of the time. 🧵

Saplings take #COLM2025! Featuring Group lunch, amazing posters, and a panel with Yoshua Bengio!

We are launching our Graduate School Application Financial Aid Program (www.queerinai.com/grad-app-aid) for 2025-2026. We’ll give up to $750 per person to LGBTQIA+ STEM scholars applying to graduate programs. Apply at openreview.net/group?id=Que.... 1/5

I'm also giving a talk at #COLM2025 Social Simulation workshop (sites.google.com/view/social-...) on Unlocking Social Intelligence in AI, at 2:30pm Oct 10th!

Social Simulation with LLMs

Important Information Workshop Date: October 10, 2025 Workshop Room: 523AB Workshop Time Zone: Montreal (UTC -4) Contact Email: social-simulation@googlegroups.com

sites.google.com

Maarten Sap@maartensap.bsky.social · last yr.

Headed to #COLM2025 today! Here's five of our papers that were accepted, and when & where to catch them 👇

Headed to #COLM2025 today! Here's five of our papers that were accepted, and when & where to catch them 👇

📢 New #COLM2025 paper 📢 Standard benchmarks give every LLM the same questions. This is like testing 5th graders and college seniors with *one* exam! 🥴 Meet Fluid Benchmarking, a capability-adaptive eval method delivering lower variance, higher validity, and reduced cost. 🧵

Ai2@ai2.bsky.social · last yr.

🚀 Introducing Fluid Benchmarking—an adaptive way to evaluate LLMs. Inspired by psychometrics, it tailors which questions to ask based on each model’s capability, making evals more efficient & reliable. 🧵

That's a lot of people! Fall Sapling lab outing, welcoming our new postdoc Vasudha, and visitors Tze Hong and Chani! (just missing Jocelyn)

I'm excited cause I'm teaching/coordinating a new unique class, where we teach new PhD students all the "soft" skills of research, incl. ideation, reviewing, presenting, interviewing, advising, etc. Each lecture is taught by a different LTI prof! It takes a village! maartensap.com/11705/Fall20...

Language Technologies Institute | CMU@ltiatcmu.bsky.social · last yr.

Happy first day of classes to all our students, new and returning! Wishing everyone a happy and enlightening fall semester.

I spoke to Forbes about why model "welfare" is a silly framing to an important issue; models don't have feelings, and it's a big distraction from real questions like tensions between safety vs. user utility, which are NLP/HCI/policy questions www.forbes.com/sites/victor...

Super super excited about this :D :D

Language Technologies Institute | CMU@ltiatcmu.bsky.social · last yr.

A hearty congratulations to the LTI's @maartensap.bsky.social, who's been awarded an Okawa Research Grant for his work in his work in socially-aware artificial intelligence. lti.cmu.edu/news-and-eve...

Hand gestures are a major mode of human communication, but they don't always translate well across cultures. New research from @akhilayerukola.bsky.social, @maartensap.bsky.social and others is aimed at giving AI systems a hand with overcoming cultural biases: lti.cmu.edu/news-and-eve...