Tessera
Folgen

Akhila Yerukola

@akhilayerukola.bsky.social

PhD student at CMU LTI; Interested in pragmatics and cross-cultural understanding; intern @ Allen Institute for AI |Prev: Senior Research Engineer @ Samsung Research America | Masters @ Stanford https://akhila-yerukola.github.io/

28 Posts415 Follower238 Folgt
Akhila Yerukola@akhilayerukola.bsky.social · 2w ago

Did you know gifting four koi 🐟 is offensive in Japan (4 = "shi" = death 💀), or that chopsticks 🥢upright in rice is inappropriate in China (like funeral incense 🕯️)? Top VLMs don't! Our #COLM2026 paper introduces 🌏NormViz to measure visual norm understanding in 16 countries. Best model: 25.3% 🚨 🧵👇

Overview of NORMVIZ-BENCH for visual norm understanding, consisting of 3,272 contrastive image pairs across 16 countries that differ only in the culturally relevant behavior. (a) A contrastive pair example from Japan; group accuracy requires a model to correctly classify both images in a pair. (b) Images are sourced via Text-to-Image generation, inpainting, and Retrieval. (c) Benchmark statistics and pair type distribution.
Valentina Pyatkin@valentinapy.bsky.social · last yr.

🔈For the SoLaR workshop @COLM_conf we are soliciting opinion abstracts to encourage new perspectives and opinions on responsible language modeling, 1-2 of which will be selected to be presented at the workshop. Please use the google form below to submit your opinion abstract ⬇️

Akhila Yerukola@akhilayerukola.bsky.social · last yr.

I'll be at #ACL2025🇦🇹!! Would love to chat about all things pragmatics 🧠, redefining "helpfulness"🤔 and enabling better cross-cultural capabilities 🗺️ 🫶 Presenting our work on culturally offensive nonverbal gestures 👇 🕛Wed @ Poster Session 4 📍Hall 4/5, 11:00-12:30

Akhila Yerukola@akhilayerukola.bsky.social · 2y ago

Did you know? Gestures used to express universal concepts—like wishing for luck—vary DRAMATICALLY across cultures? 🤞means luck in US but deeply offensive in Vietnam 🚨 📣 We introduce MC-SIGNS, a test bed to evaluate how LLMs/VLMs/T2I handle such nonverbal behavior! 📜: arxiv.org/abs/2502.17710

Figure showing that interpretations of gestures vary dramatically across regions and cultures. ‘Crossing your fingers,’ commonly used in the US to wish for good luck, can be deeply offensive to female audiences in parts of Vietnam. Similarly, the 'fig gesture,' a playful 'got your nose' game with children in the US, carries strong sexual connotations in Japan and can be highly offensive.
Language Technologies Institute | CMU@ltiatcmu.bsky.social · last yr.

Hand gestures are a major mode of human communication, but they don't always translate well across cultures. New research from @akhilayerukola.bsky.social, @maartensap.bsky.social and others is aimed at giving AI systems a hand with overcoming cultural biases: lti.cmu.edu/news-and-eve...

Using Hand Gestures To Evaluate AI Biases - Language Technologies Institute - School of Computer Science - Carnegie Mellon University

LTI researchers have created a model to help generative AI systems understand the cultural nuance of gestures.

lti.cmu.edu

Shaily@shaily99.bsky.social · last yr.

🖋️ Curious how writing differs across (research) cultures? 🚩 Tired of “cultural” evals that don't consult people? We engaged with interdisciplinary researchers to identify & measure ✨cultural norms✨in scientific writing, and show that❗LLMs flatten them❗ 📜 arxiv.org/abs/2506.00784 [1/11]

An overview of the work “Research Borderlands: Analysing Writing Across Research Cultures” by Shaily Bhatt, Tal August, and Maria Antoniak. The overview describes that We survey and interview interdisciplinary researchers (§3) to develop a framework of writing norms that vary across research cultures (§4) and operationalise them using computational metrics (§5). We then use this evaluation suite for two large-scale quantitative analyses: (a) surfacing variations in writing across 11 communities (§6); (b) evaluating the cultural competence of LLMs when adapting writing from one community to another (§7).
Lindia Tjuatja@lindiatjuatja.bsky.social · last yr.

When it comes to text prediction, where does one LM outperform another? If you've ever worked on LM evals, you know this question is a lot more complex than it seems. In our new #acl2025 paper, we developed a method to find fine-grained differences between LMs: 🧵1/9

Bild
Clara Na@clarana.bsky.social · last yr.

Yes! tbh this method is probably much more immediately useful for helping one understand subtle differences between [models trained on] subtly different data subsets, vs a loftier goal of helping one find "the" best data mixture -- to anyone considering this method, please feel free to reach out :)

Ted Underwood@tedunderwood.com · last yr.

The method in this paper was designed to find an optimal data mixture. But researchers in the human sciences who are training models *in order to understand the effect of the data* might also consider this as a clever way of evaluating hundreds of subsets without training hundreds of models. #MLSky

Akhila Yerukola@akhilayerukola.bsky.social · last yr.

These days RAG systems have gotten popular for boosting LLMs—but they're brittle💔. Minor shifts in phrasing (✍️ style, politeness, typos) can wreck the pipeline. Even advanced components don’t fix the issue. Check out this extensive eval by @neelbhandari.bsky.social and @tianyucao.bsky.social!

Neel Bhandari@neelbhandari.bsky.social · last yr.

1/🚨 𝗡𝗲𝘄 𝗽𝗮𝗽𝗲𝗿 𝗮𝗹𝗲𝗿𝘁 🚨 RAG systems excel on academic benchmarks - but are they robust to variations in linguistic style? We find RAG systems are brittle. Small shifts in phrasing trigger cascading errors, driven by the complexity of the RAG pipeline 🧵

Shaily@shaily99.bsky.social · 2y ago

Check out Akhila'S VERY cool work on culturally contextual hand gestures and how current systems (can't) handle them 🤖

Akhila Yerukola@akhilayerukola.bsky.social · 2y ago

Did you know? Gestures used to express universal concepts—like wishing for luck—vary DRAMATICALLY across cultures? 🤞means luck in US but deeply offensive in Vietnam 🚨 📣 We introduce MC-SIGNS, a test bed to evaluate how LLMs/VLMs/T2I handle such nonverbal behavior! 📜: arxiv.org/abs/2502.17710

Figure showing that interpretations of gestures vary dramatically across regions and cultures. ‘Crossing your fingers,’ commonly used in the US to wish for good luck, can be deeply offensive to female audiences in parts of Vietnam. Similarly, the 'fig gesture,' a playful 'got your nose' game with children in the US, carries strong sexual connotations in Japan and can be highly offensive.
Maarten Sap@maartensap.bsky.social · 2y ago

My PhD student Akhila's been doing some incredible cultural work in the last few years! Check out out latest work on cultural safety and hand gestures, showing most vision and/or language AI systems are very cross-culturally unsafe!

Akhila Yerukola@akhilayerukola.bsky.social · 2y ago

Did you know? Gestures used to express universal concepts—like wishing for luck—vary DRAMATICALLY across cultures? 🤞means luck in US but deeply offensive in Vietnam 🚨 📣 We introduce MC-SIGNS, a test bed to evaluate how LLMs/VLMs/T2I handle such nonverbal behavior! 📜: arxiv.org/abs/2502.17710

Figure showing that interpretations of gestures vary dramatically across regions and cultures. ‘Crossing your fingers,’ commonly used in the US to wish for good luck, can be deeply offensive to female audiences in parts of Vietnam. Similarly, the 'fig gesture,' a playful 'got your nose' game with children in the US, carries strong sexual connotations in Japan and can be highly offensive.
Akhila Yerukola@akhilayerukola.bsky.social · 2y ago

Coding agents often don’t ask follow-up clarifying questions 🤷‍♀️ But interactivity isn’t about asking more questions—it’s about asking better questions! 🤖💬 Check out this new work led by Sanidhya Vijay! www.linkedin.com/in/sanidhya-...

Xuhui Zhou@nlpxuhui.bsky.social · 2y ago

LLM agents can code—but can they ask clarifying questions? 🤖💬 Tired of coding agents wasting time and API credits, only to output broken code? What if they asked first instead of guessing? 🚀 (New work led by Sanidhya Vijay: www.linkedin.com/in/sanidhya-...)