Martin Tutek

@mtutek.bsky.social

Postdoc @ TakeLab, UniZG | previously: Technion; TU Darmstadt | PhD @ TakeLab, UniZG Faithful explainability, controllability & safety of LLMs. 🔎 On the academic job market 🔎 https://mttk.github.io/

⏳ The BlackboxNLP 2026 Reproducibility Challenge deadline has been extended to July 24 (AoE) ⏳ If you've been working on a robustness check, ablation, or replication of recent NLP interpretability work, you have a bit more time to get your submission in.

Bild

With the large influx of submissions and a faster pace of research, reproducibility is more important than ever. With this reproducibility challenge, we want to put the focus on best practices wrt. baselines🧱, ablations🌈, eval🔎 and generalizability🗺️ of interpretability!

BlackboxNLP@blackboxnlp.bsky.social · 2mo ago

📣 Announcing the BlackboxNLP 2026 Reproducibility Challenge! A new track dedicated to rigorous robustness checks of NLP interpretability work - stress-testing baselines, ablations, generalizability, and evaluation.

📣 Announcing the BlackboxNLP 2026 Reproducibility Challenge! A new track dedicated to rigorous robustness checks of NLP interpretability work - stress-testing baselines, ablations, generalizability, and evaluation.

Bild

Ever used a top-ranked LLM that just... felt wrong for you? You’re not alone. Instead of leaderboards, many of us turn to "vibe-testing" - manually comparing models to our own needs. But can we turn these feelings into a structured evaluation? New paper: "From Feelings to Metrics" 🧵

Bild

❗The full paper submission deadline for COLM is ~14 hours from now (11:59pm AOE)! Please submit your final PDFs on the same page where you uploaded your abstracts. And please use the provided LaTeX templates; do not handwrite your manuscript like this llama is! Good luck!

A llama sweating while writing a paper at a desk. A sign says "Deadline! March 31 11:59pm AOE"

To ease my FOMO from not attending @eaclmeeting.bsky.social, I skimmed the proceedings while playing the Tangier episode of Parts Unknown. I'll do something different and shout out 5 (subjectively) interesting works of authors I'm *not* closely related to, in no specific order:🧵⬇️

Excited to present this work together with @dippedrusk.com at #EACL. Join us in the poster session 1 (11:30-13:00) 🔥

Poster of the Paper Aligned Probing
Andreas Waldis@tresiwald.bsky.social · 6mo ago

LMs that "know more" about toxicity are less toxic! Our #TACL 📄 connects behavior and internals: 💠 LMs amplify toxicity beyond humans 💠 Information about toxicity peaks in lower layers 💠 Bypassing these layers increases toxicity More details👇 #NLProc #interpretability (1/🧵)

simplified overview of our aligned probing setup, where we join the behavioral and internal evaluation of LMs' toxicity

We'll have a reproducibility track at this years' Blackbox workshop! Details are still within a slightly opaque box. We want to see if cleaning solutions that make opaque boxes 📦 transparent 🍱 work on different boxes 🎁📮🧰🥡, and with different 🧽solution-to-water🧼ratios!

BlackboxNLP@blackboxnlp.bsky.social · 5mo ago

BlackboxNLP will be co-located with EMNLP 2026 in 🇭🇺 Budapest 🇭🇺 this October! This edition will feature a special reproducibility track, investigating generalization and robustness of established results from interpretability research 👷‍♂️ Stay tuned for more details!