Super cool project that I really enjoyed being part of! tl;dr - when a human or model encounters new visual stimuli, how closely is it mapped to other, previously encountered concepts? (Come for weird dog-monster, stay for the science 🙂 )

Ada@adadtur.bsky.social · 4mo ago

Super excited to finally announce my latest research “Would you still call this Dax? Novel Visual References in VLMs and Humans”! We studied how vision-language models (VLMs) adopt new visual concepts and map them to language compared to humans, and found that…

My first last-author paper is out! If you saw this dog below and someone showed you the second image, would you consider them the same word/concept? (more examples in Ada's thread) We study if VLMs agree with humans on this and revisit old questions around shape vs. texture bias in vision

Ada@adadtur.bsky.social · 4mo ago

Super excited to finally announce my latest research “Would you still call this Dax? Novel Visual References in VLMs and Humans”! We studied how vision-language models (VLMs) adopt new visual concepts and map them to language compared to humans, and found that…

Super excited to finally announce my latest research “Would you still call this Dax? Novel Visual References in VLMs and Humans”! We studied how vision-language models (VLMs) adopt new visual concepts and map them to language compared to humans, and found that…

Takeaway: reasoning LLMs are getting better and better on math and code—deterministic reasoning tasks. But we should also evaluate them on open-ended, inherently uncertain everyday reasoning! (9/10)

🚨New Paper!🚨 How do reasoning LLMs handle inferences that have no deterministic answer? We find that they diverge from humans in some significant ways, and fail to reflect human uncertainty… 🧵(1/10)

Beyond our controlled setup, we also show how LatentLens works much better than baselines on off-the-shelf Qwen2-VL-7B-Instruct

Building a VLM can be surprisingly simple: You keep both the LLM and vision encoder frozen, you just train a small MLP that projects into the LLM embedding space as prefixes. That’s it 😮 But how and why does that work? How do visual tokens relate to language, i.e. do they have interpretable NNs?

Paper: arxiv.org/abs/2602.00462 Code: github.com/McGill-NLP/... Demo: tinyurl.com/ce57mn4v Couldn't have imagined better collaborators to wrap up the phd: Shravan Nayak @oscmansan.bsky.social @vaibhavadlakha.bsky.social @delliott.bsky.social @sivareddyg.bsky.social @mariusmosbach.bsky.social

GitHub - McGill-NLP/latentlens: Code and data for the paper "LatentLens: Revealing Highly Interpretable Visual Tokens in LLMs"

Code and data for the paper "LatentLens: Revealing Highly Interpretable Visual Tokens in LLMs" - McGill-NLP/latentlens

github.com

🚨New paper Are visual tokens going into an LLM interpretable 🤔 Existing methods (e.g. logit lens) and assumptions would lead you to think “not much”... We propose LatentLens and show that most visual tokens are interpretable across *all* layers 💡 Details 🧵

"Not only is the ratio of AI’s resource rapacity to its productive utility indefensibly and irremediably skewed, AI-made material is itself a waste product: flimsy, shoddy, disposable, a single-use plastic of the mind." >>

enshittification | noun | when a digital platform is made worse for users, in order to increase profits

Look what they did to Notepad. Shut the fuck up. This is Notepad. You are not welcome here. Oh yeah "Let me use Copilot for Notepad". "I'm going to sign into my account for Notepad". What the fuck are you talking about. It's Notepad.

Our new paper in #PNAS (bit.ly/4fcWfma) presents a surprising finding—when words change meaning, older speakers rapidly adopt the new usage; inter-generational differences are often minor. w/ Michelle Yang, ‪@sivareddyg.bsky.social‬ , @msonderegger.bsky.social‬ and @dallascard.bsky.social‬👇(1/12)

Thrilled to announce our new survey that explores the exciting possibilities and troubling risks of computational persuasion in the era of LLMs 🤖💬 📄Arxiv: arxiv.org/pdf/2505.07775 💻 GitHub: github.com/beyzabozdag/...

Started a new podcast with @tomvergara.bsky.social ! Behind the Research of AI: We look behind the scenes, beyond the polished papers 🧐🧪 If this sounds fun, check out our first "official" episode with the awesome Gauthier Gidel from @mila-quebec.bsky.social : open.spotify.com/episode/7oTc...

02 | Gauthier Gidel: Bridging Theory and Deep Learning, Vibes at Mila, and the Effects of AI on Art

Behind the Research of AI · Episode

open.spotify.com

Zohran Mamdani, a 33-year-old state assemblyman, declared victory in New York City’s Democratic mayoral primary after Andrew Cuomo conceded the race. “Tonight we made history,” Mamdani said, addressing his supporters. wapo.st/44yMVoI

Mahmoud Khalil is finally home with his beautiful wife and newborn son. Each one of the 104 days he spent detained was a grave injustice. From the moment of his detention, @ccrjustice.org + @aclu.org engaged my office as we worked closely to help secure his release. They did remarkable work here.

Gwynne Hogan@gwynnefitz.bsky.social · last yr.

Mahmoud Khalil emerges from airport security with his wife Noor and newborn, legal team and Rep. AOC.

The facts: We release (MVPBench) with around 55K videos (grouped as *minimal video pairs*) from diverse physical understanding sources Arxiv: arxiv.org/abs/2506.09987 Huggingface: huggingface.co/datasets/fac... GitHub: github.com/facebookrese... Leaderboard: huggingface.co/spaces/faceb...

A Shortcut-aware Video-QA Benchmark for Physical Understanding via Minimal Video Pairs

Existing benchmarks for assessing the spatio-temporal understanding and reasoning abilities of video language models are susceptible to score inflation due to the presence of shortcut solutions based ...

arxiv.org

Excited to share the results of my recent internship! We ask 🤔 What subtle shortcuts are VideoLLMs taking on spatio-temporal questions? And how can we instead curate shortcut-robust examples at a large-scale? We release: MVPBench Details 👇🔬

Today, I was denied access to seeing my constituent, Mr. Kilmar Abrego Garcia. If there is nothing to hide, cut the crap. Let his lawyer and I check on him.

Breaking news: The Trump administration revoked Harvard’s ability to enroll foreign students, saying it allowed anti-American agitators. Existing foreign students must transfer or risk losing their legal status, DHS said.

Live updates: Trump administration revokes Harvard’s ability to enroll foreign students

Get the latest news on President Donald Trump’s return to the White House and the Republican-led Congress.

wapo.st

when in albuquerque…

We won a Senior Area Chair Award at NAACL!! Many thanks again to my amazing coauthors Gaurav Kamath and @sivareddyg.bsky.social :-)

Ada@adadtur.bsky.social · 2y ago

Super excited to finally announce our NAACL 2025 main conference paper “Language Models Largely Exhibit Human-like Constituent Ordering Preferences”! We examine constituent ordering preferences between humans and LLMs; we present two main findings… 🧵

Check out Gaurav's video on their #NAACL paper and find @adadtur.bsky.social at the conference 👇

Mila - Institut québécois d'IA@mila-quebec.bsky.social · last yr.

Congratulations to Mila members @adadtur.bsky.social , Gaurav Kamath and @sivareddyg.bsky.social for their SAC award at NAACL! Check out Ada's talk in Session I: Oral/Poster 6. Paper: arxiv.org/abs/2502.05670

Great work from labmates on LLMs vs humans regarding linguistic preferences: You know when a sentence kind of feels off e.g. "I met at the park the man". So in what ways do LLMs follow these human intuitions?

Mila - Institut québécois d'IA@mila-quebec.bsky.social · last yr.

Congratulations to Mila members @adadtur.bsky.social , Gaurav Kamath and @sivareddyg.bsky.social for their SAC award at NAACL! Check out Ada's talk in Session I: Oral/Poster 6. Paper: arxiv.org/abs/2502.05670

Ada is an undergrad and will soon be looking for PhDs. Gaurav is a PhD student looking for intellectually stimulating internships/visiting positions. They did most of the work without much of my help. Highly recommend them. Please reach out to them if you have any positions.

Language Models Largely Exhibit Human-like Constituent Ordering Preferences

Though English sentences are typically inflexible vis-à-vis word order, constituents often show far more variability in ordering. One prominent theory presents the notion that constituent ordering is ...

arxiv.org

Incredibly proud of my students @adadtur.bsky.social and Gaurav Kamath for winning a SAC award at #NAACL2025 for their work on assessing how LLMs model constituent shifts.

Language Models Largely Exhibit Human-like Constituent Ordering Preferences

Though English sentences are typically inflexible vis-à-vis word order, constituents often show far more variability in ordering. One prominent theory presents the notion that constituent ordering is ...

arxiv.org

Mila - Institut québécois d'IA@mila-quebec.bsky.social · last yr.

Congratulations to Mila members @adadtur.bsky.social , Gaurav Kamath and @sivareddyg.bsky.social for their SAC award at NAACL! Check out Ada's talk in Session I: Oral/Poster 6. Paper: arxiv.org/abs/2502.05670

Congratulations to Mila members @adadtur.bsky.social , Gaurav Kamath and @sivareddyg.bsky.social for their SAC award at NAACL! Check out Ada's talk in Session I: Oral/Poster 6. Paper: arxiv.org/abs/2502.05670

I filmed this yesterday on my way to Lousiana where my constituent Rümeysa Öztürk is being wrongfully held by ICE. I’m there now demanding her release. More to come.