Martin Tutek

@mtutek.bsky.social

Postdoc @ TakeLab, University of Zagreb | Working on interpretability & safety of LLMs. https://mttk.github.io/

Models are sycophantic, and when they do challenge faulty or incorrect inputs, they're ambiguous and ineffective. In a new paper w/ a multilingual co-author team, we find that people may view LM responses that push back, yet still end up believing false content is true. arxiv.org/abs/2610.17703

A screenshot of the top half of the first page of a paper titled "Pushback in a Haystack: Language Models Respond to Factual Errors Ambiguously and Ineffectively" with authors Li Lucy, Myra Cheng, Svenja Guhr, Kabir Ahuja, Orevaoghene Ahia, Sameep Chattopadhyay, Yike Wang, Valentin Hofmann, Noah A. Smith, Yulia Tsvetkov

Our (objectively awesome) program is now live! Join us on October 29th for an 🎢 interpretaganza🎢 in Budapest. Come and enjoy the great talks, insightful conversations and a surprising lack of slop (except for coffee*)☕️ *jk I am actually ok with conference coffee

BlackboxNLP@blackboxnlp.bsky.social · 3w ago

📣 The BlackboxNLP 2026 program is live! Join us on October 29th in Room P3 at @emnlpmeeting.bsky.social for keynotes by @sfeucht.bsky.social , Ivan Titov & @m-hahn.bsky.social, six oral presentations, and two poster sessions! Full schedule: blackboxnlp.github.io/2026/#program

Due to the unprecedented amount of submissions to BlackboxNLP 2026, we will announce the decisions for Main, Special and ARR commitments on September 2 AoE. Thanks for your patience!

⏳ The BlackboxNLP 2026 Reproducibility Challenge deadline has been extended to July 24 (AoE) ⏳ If you've been working on a robustness check, ablation, or replication of recent NLP interpretability work, you have a bit more time to get your submission in.

Bild

With the large influx of submissions and a faster pace of research, reproducibility is more important than ever. With this reproducibility challenge, we want to put the focus on best practices wrt. baselines🧱, ablations🌈, eval🔎 and generalizability🗺️ of interpretability!

BlackboxNLP@blackboxnlp.bsky.social · 5mo ago

📣 Announcing the BlackboxNLP 2026 Reproducibility Challenge! A new track dedicated to rigorous robustness checks of NLP interpretability work - stress-testing baselines, ablations, generalizability, and evaluation.

📣 Announcing the BlackboxNLP 2026 Reproducibility Challenge! A new track dedicated to rigorous robustness checks of NLP interpretability work - stress-testing baselines, ablations, generalizability, and evaluation.

Bild

Ever used a top-ranked LLM that just... felt wrong for you? You’re not alone. Instead of leaderboards, many of us turn to "vibe-testing" - manually comparing models to our own needs. But can we turn these feelings into a structured evaluation? New paper: "From Feelings to Metrics" 🧵

Bild

❗The full paper submission deadline for COLM is ~14 hours from now (11:59pm AOE)! Please submit your final PDFs on the same page where you uploaded your abstracts. And please use the provided LaTeX templates; do not handwrite your manuscript like this llama is! Good luck!

A llama sweating while writing a paper at a desk. A sign says "Deadline! March 31 11:59pm AOE"

To ease my FOMO from not attending @eaclmeeting.bsky.social, I skimmed the proceedings while playing the Tangier episode of Parts Unknown. I'll do something different and shout out 5 (subjectively) interesting works of authors I'm *not* closely related to, in no specific order:🧵⬇️