New Paper: arxiv.org/abs/2606.19370 Self-play yields capabilities but requires frustrating cost-function tuning. Surprisingly, just 30 minutes of demonstration data produces much more human-like driving policies! Led by @daphne-cornelisse.bsky.social Website: spiced-self-play.com

#CVPR2026 paper: It's Never Too Late: Noise Optimization for Collapse Recovery in Trained Diffusion Models Text-to-image models often collapse to near-identical samples. Our fix: optimize the noise. Start from pink 🩷, not white noise. 🔗 akoepke.github.io/divgen/index... 1/6

There's nothing more satisfying than watching the right noise do its magic with diffusion models! A few interesting takeaways I had from this work 🧵

A. Sophia Koepke@askoepke.bsky.social · 2mo ago

#CVPR2026 paper: It's Never Too Late: Noise Optimization for Collapse Recovery in Trained Diffusion Models Text-to-image models often collapse to near-identical samples. Our fix: optimize the noise. Start from pink 🩷, not white noise. 🔗 akoepke.github.io/divgen/index... 1/6

👋🇧🇷 If you are at #ICLR2026 today, you should talk to @antonbaumann.bsky.social who is presenting our paper about turning pre-trained VLMs into probabilistic models without retraining or fine-tuning. Poster Session 3 ⌚: 10:30am - 1:00pm (local time) 📍: Pavilion 3 P3 - #313 @iclr-conf.bsky.social

New paper: Back into Plato’s Cave Are vision and language models converging to the same representation of reality? The Platonic Representation Hypothesis says yes. BUT we find the evidence for this is more fragile than it looks. Project page: akoepke.github.io/cave_umwelten/ 1/9

Back into Plato's Cave: Examining Cross-modal Representational Convergence at Scale

Back into Plato's Cave: Examining Cross-modal Representational Convergence at Scale

akoepke.github.io

Hey everyone, super happy to share our work on quantum algorithms for heterogeneous partial differential equations (PDEs)! (1/4) scirate.com/arxiv/2604.0...

This has now been accepted at @iclr-conf.bsky.social !

Martin Trapp@trappmartin.eurosky.social · 11mo ago

Unfortunately, our submission to #NeurIPS didn’t go through with (5,4,4,3). But because I think it’s an excellent paper, I decided to share it anyway. We show how to efficiently apply Bayesian learning in VLMs, improve calibration, and do active learning. Cool stuff! 📝 arxiv.org/abs/2412.06014

Was a very fun (and quick) investigation into biases of multimodal benchmarks, this time on tasks designed for "Spatial Supersensing" introduced by Cambrian-S with some great folks!

Andreas Hochlehnert@ahochlehnert.bsky.social · 8mo ago

🚨 New Paper: "Solving Spatial Supersensing Without Spatial Supersensing" Huge credit to the Cambrian-S team for tackling one of the hardest open problems in video understanding: spatial supersensing. In our paper, we take a closer look at their benchmarks & methods 👇

🚨 New Paper: "Solving Spatial Supersensing Without Spatial Supersensing" Huge credit to the Cambrian-S team for tackling one of the hardest open problems in video understanding: spatial supersensing. In our paper, we take a closer look at their benchmarks & methods 👇

Unfortunately, our submission to #NeurIPS didn’t go through with (5,4,4,3). But because I think it’s an excellent paper, I decided to share it anyway. We show how to efficiently apply Bayesian learning in VLMs, improve calibration, and do active learning. Cool stuff! 📝 arxiv.org/abs/2412.06014

Post-hoc Probabilistic Vision-Language Models

Vision-language models (VLMs), such as CLIP and SigLIP, have found remarkable success in classification, retrieval, and generative tasks. For this, VLMs deterministically map images and text descripti...

arxiv.org

Wonderful story behind some very nice SSL work!

Andrei Bursuc@abursuc.bsky.social · last yr.

1/ New & old work on self-supervised representation learning (SSL) with ViTs: MOCA ☕ - Predicting Masked Online Codebook Assignments w/ @spyrosgidaris.bsky.social O. Simeoni, A. Vobecky, @matthieucord.bsky.social, N. Komodakis, @ptrkprz.bsky.social #TMLR #ICLR2025 Grab a ☕ & brace for a story & a🧵

I'm in Nashville this week attending #CVPR2025. Excited to discuss post-training VLMs and diffusion models!

We're super happy: Our Cluster of Excellence will continue to receive funding from the German Research Foundation @dfg.de ! Here’s to 7 more years of exciting research at the intersection of #machinelearning and science! Find out more: uni-tuebingen.de/en/research/... #ExcellenceStrategy

🚨 New preprint! How far can we go with ImageNet for Text-to-Image generation? w. @arrijitghosh.bsky.social @lucasdegeorge.bsky.social @nicolasdufour.bsky.social @vickykalogeiton.bsky.social TL;DR: Train a text-to-image model using 1000 less data in 200 GPU hrs! 📜https://arxiv.org/abs/2502.21318 🧵👇

These are some ridiculously good results from training tiny T2I models purely on ImageNet! It's almost too good to be true. Do check it out!

Nicolas Dufour@nicolasdufour.bsky.social · last yr.

Check out our latest work on Text-to-Image generation! We've successfully trained a T2I model using only ImageNet data by leveraging captioning and data augmentation.

I've been talking about writing this paper to anyone who would listen since 2020. I bombed a bunch of job talks trying to convince companies to work on this. It's so nice to finally just be able to say, yes, self-play RL in a diverse world gives you immense capabilities arxiv.org/abs/2502.03349

Robust Autonomy Emerges from Self-Play

Self-play has powered breakthroughs in two-player and multi-player games. Here we show that self-play is a surprisingly effective strategy in another domain. We show that robust and naturalistic drivi...

arxiv.org

🚨Great Models Think Alike and this Undermines AI Oversight🚨 New paper quantifies LM similarity (1) LLM-as-a-judge favor more similar models🤥 (2) Complementary knowledge benefits Weak-to-Strong Generalization☯️ (3) More capable models have more correlated failures 📈🙀 🧵👇

ReNO shows that some initial noise are better for some prompts! This is great to improve image generation, but i think it also shows a deeper property of diffusion models.

Luca Eyring@lucaeyring.bsky.social · 2y ago

Can we enhance the performance of T2I models without any fine-tuning? We show that with our ReNO, Reward-based Noise Optimization, one-step models consistently surpass the performance of all current open-source Text-to-Image models within the computational budget of 20-50 sec! #NeurIPS2024

This is maybe my favorite thing I've seen out of #NeurIPS2024. Head over to HuggingFace and play with this thing. It's quite extraordinary.

Luca Eyring@lucaeyring.bsky.social · 2y ago

Thanks to @fffiloni.bsky.social and @natanielruiz.bsky.social, we have a running live Demo of ReNO, play around with it here: 🤗: huggingface.co/spaces/fffil... We are excited to present ReNO at #NeurIPS2024 this week! Join us tomorrow from 11am-2pm at East Exhibit Hall A-C #1504!

Can we enhance the performance of T2I models without any fine-tuning? We show that with our ReNO, Reward-based Noise Optimization, one-step models consistently surpass the performance of all current open-source Text-to-Image models within the computational budget of 20-50 sec! #NeurIPS2024

I will present ✌️ BDU workshop papers @ NeurIPS: one by Rui Li (looking for internships) and one by Anton Baumann. 🔗 to extended versions: 1. 🙋 "How can we make predictions in BDL efficiently?" 👉 arxiv.org/abs/2411.18425 2. 🙋 "How can we do prob. active learning in VLMs" 👉 arxiv.org/abs/2412.06014

Post-hoc Probabilistic Vision-Language Models

Vision-language models (VLMs), such as CLIP and SigLIP, have found remarkable success in classification, retrieval, and generative tasks. For this, VLMs deterministically map images and text descripti...

arxiv.org

Martin Trapp@trappmartin.eurosky.social · 2y ago

On my way to #NeurIPS. Looking forward to seeing many friends again. Ping me if you want to chat, always happy to meet new people. :)

After a break of over 2 years, I'm attending a conference again! Excited to attend NeurIPS, even more so to be presenting ReNO, getting inference-time scaling and preference optimization to work for text-to-image generation. Do reach out if you'd like to chat!

🚨New Paper Alert🚨 🚀 Introducing FlowChef, "Steering Rectified Flow Models in the Vector Field for Controlled Image Generation"! 🌌✨ - Perform image editing, solve inverse problems, and more. - Achieved inversion-free, gradient-free, & training-free inference time steering! 🤯 👇👇

Some recent discussions made me write up a short read on how I think about doing computer vision research when there's clear potential for abuse. Alternative title: why I decided to stop working on tracking. Curious about other's thoughts on this. lb.eyer.be/s/cv-ethics....

Check out this nice work by @confusezius.bsky.social on designing VLMs for few-shot adaptation!

Karsten Roth@confusezius.bsky.social · 2y ago

🤔 Can you turn your vision-language model from a great zero-shot model into a great-at-any-shot generalist? Turns out you can, and here is how: arxiv.org/abs/2411.15099 Really excited to this work on multimodal pretraining for my first bluesky entry! 🧵 A short and hopefully informative thread:

A real-time (or very fast) open-source txt2video model dropped: LTXV. HF: huggingface.co/Lightricks/L... Gradio: huggingface.co/spaces/Light... Github: github.com/Lightricks/L... Look at that prompt example though. Need to be a proper writer to get that quality.

Learning from one continuous video stream - use a video stream to learn a predictive model - everything is in pixel space - update the model less frequently and don’t use momentum optimizer - pre training with iid improves performance - continual learning for robots arxiv.org/html/2312.00...

Here's a fledgling starter pack for the AI community in Tübingen. Let me know if you'd like to be added! go.bsky.app/NFbVzrA

Tübingen AI

Join the conversation

go.bsky.app