Check out our new paper, investigating phenomena (hallucination, refusal, and sycophancy) both externally and internally! Showing a high correlation between the two!

Martin Tutek@mtutek.bsky.social · 5mo ago

How does an LLM’s past influence its future?🤔 In new work, led by @adisimhi.bsky.social, together with @fbarez.bsky.social @boknilev.bsky.social and Shay Cohen, we find conversational history creates a latent "geometric trap" which makes old habits e.g. hallucinations hard to break!

ManagerBench was accepted to #ICLR2026🎉 Check it out⬇️

Martin Tutek@mtutek.bsky.social · 7mo ago

ManagerBench was accepted to ICLR! @iclr-conf.bsky.social #ICLR2026 LLMs are still either unsafe, or completely harm avoidant - even when the harm affects furniture 🛋️ Check out our benchmark, online or in Rio 🇧🇷

Check out our new paper on evaluating LLM agents on their preference for achieving their goal and avoiding human harm, called ManagerBench👔

Martin Tutek@mtutek.bsky.social · 11mo ago

🤔What happens when LLM agents choose between achieving their goals and avoiding harm to humans in realistic management scenarios? Are LLMs pragmatic or prefer to avoid human harm? 🚀 New paper out: ManagerBench: Evaluating the Safety-Pragmatism Trade-off in Autonomous LLMs🚀🧵

🚨New arXiv preprint!🚨 LLMs can hallucinate - but did you know they can do so with high certainty even when they know the correct answer? 🤯 We find those hallucinations in our latest work with @itay-itzhak.bsky.social, @fbarez.bsky.social, @gabistanovsky.bsky.social and Yonatan Belinkov