If an LLM agent notices conflicts, will it tell you? Our #EMNLP2026 Findings paper finds that higher accuracy doesn't always mean greater epistemic humility: agents can spot conflicts early yet leave uncertainty out of wrong answers. 📄 arxiv.org/abs/2610.12360 🧵👇 #NLProc #LLM

Accurate but Not Humble: Evaluating Epistemic Humility in LLM...

When retrieved evidence contradicts an agent's prior beliefs, does it revise its answer, acknowledge uncertainty, or persist with an incorrect conclusion? Existing evaluations of agentic systems...

arxiv.org

Multimodal LLMs can read text in images, but why do they often perform worse than when the same text is given as tokens? Our work studies the modality gap of models perceiving text as pixels and shows how to close it. 📄 arxiv.org/abs/2603.09095 🧵👇 #NLProc #LLM #ComputerVision

What happens when an LLM is asked to use information that contradicts its knowledge? We explore knowledge conflict in a new preprint📑 TLDR: Performance drops, and this could affect the overall performance of LLMs in model-based evaluation.📑🧵⬇️ 1/8 #NLProc #LLM #AIResearch

What Is Seen Cannot Be Unseen: The Disruptive Effect of Knowledge Conflict on Large Language Models

Large language models frequently rely on both contextual input and parametric knowledge to perform tasks. However, these sources can come into conflict, especially when retrieved documents contradict…

arxiv.org

Had so many fruitful discussions and made many friends this #NAACL2025 🌵🏜️Thanks for everyone who came to my poster or listened to me talking about my audacious thoughts! 😜 (I should have printed more stickers as they were more popular than I anticipated😅)

Dialects lie on continua of (structured) linguistic variation, right? And we can’t collect data for every point on the continuum...🤔 📢 Check out DialUp, a technique to make your MT model robust to the dialect continua of its training languages, including unseen dialects. arxiv.org/abs/2501.16581

Meta literally created a LGBTQ exception for calling someone mentally ill as an insult. You can't do it for any other group except LGBTQ people.

with reasonable freedom, depending on the scale/focus of the business. Case in point, we are looking to expand the research/foundation models team at Orby AI and are looking for highly motivated researchers and ML/Research engineers. Please reach out if you're interested in learning more! /fin

Excited to start my #ARR #NLP reviews! I'll try my best and see if I can get 100% of my reviews to be 'great' this round. If you didn't see it already, ARR publishes how many of your reviews are considered to be 'great': stats.aclrollingreview.org Join me for the challenge :)

🚨 I am on the faculty job market this year 🚨 I will be presenting at #NeurIPS2024 and am happy to chat in-person or digitally! I work on developing AI agents that can collaborate and communicate robustly with us and each other. More at: esteng.github.io and in thread below 🧵👇

Is MMLU Western-centric? 🤔 As part of a massive cross-institutional collaboration: 🗽Find MMLU is heavily overfit to western culture 🔍 Professional annotation of cultural sensitivity data 🌍 Release improved Global-MMLU 42 languages 📜 Paper: arxiv.org/pdf/2412.03304 📂 Data: hf.co/datasets/Coh...

🚨I too am on the job market‼️🤯 I'm searching for faculty positions/postdocs in multilingual/multicultural NLP, vision+language models, and eval for genAI! I'll be at #NeurIPS2024 presenting our work on meta-evaluation for text-to-image faithfulness! Let's chat there! Papers in🧵, see more: saxon.me

Excited to share OLMo 2! 🐟 7B and 13B weights, trained up to 4-5T tokens, fully open data, code, etc 🐠 better architecture and recipe for training stability 🐡 staged training, with new data mix Dolmino🍕 added during annealing 🦈 state-of-the-art OLMo 2 Instruct models #nlp #mlsky links below👇

Putting together a JHU Center for Language and Speech Processing starter pack! Please reply or DM me if you're doing research at CLSP and would like to be added - I'm still trying to find out which of us are on here so far. go.bsky.app/JtWKca2

CLSP

Join the conversation

go.bsky.app

A starter pack for #NLP #NLProc researchers! 🎉 go.bsky.app/SngwGeS

Dealing with a new social media account can be vexatious. Here I compiled a thread of resources that might be helpful to transition to Bluesky 🦋 . 🧵⬇️ Thread below