🚨 You are only evaluating a slice of your test-time scaling model's performance! 🚨 📈 We consider how models’ confidence in their answers changes as test-time compute increases. Reasoning longer helps models answer more confidently! 📝: arxiv.org/abs/2502.13962

New Workshop on Multimodal Augmented Generation via MultimodAl Retrieval (MAGMaR) to be held at @aclmeeting.bsky.social ACL in Vienna this summer. We have a new shared task that stumps most LLMs - including ones pretrained on our test collection. nlp.jhu.edu/magmar/

The ML/NLP grad student starter packs grew fast! I had to make a second one. Here's a list you can use to view combined posts from both packs: bsky.app/profile/marc... You can "Pin to home" to see it as a tab. Looks like students are getting ready for NeurIPS soon 👀

I noticed a lot of starter packs skewed towards faculty/industry, so I made one of just NLP & ML students: go.bsky.app/vju2ux Students do different research, go on the job market, and recruit other students. Ping me and I'll add you!

Meeting notes with Cody this week: "do you think factorio space age was a psyop for ai slowdown?"

Cody Blakeney ✈️ NeurIPS 2024@codestar.bsky.social · 2y ago

I just installed factorio. I’m told I might not be seen again for a while.