Instead of actually generating CoT, this paper uses a hypernetwork to predict small specific updates to the LLM's bias parameters, approximating the effect of thinking without generating the trace. On some benchmarks, it matches thinking with only 7% of the latency alphaxiv.org/abs/2610.03039

I'm heading to @colmweb.org. Come find me to chat about tokenizers and Korean language modeling! I'll be presenting a poster at @tokshop.bsky.social about the need for a community tokenization benchmark a la the NanoGPT speedrun. It is meant to be a discussion, come share your ideas! #colm26

Running more loops than they were trained on usually hurts generation quality for looped transformers This paper LiFT fixes this by training each loop to progressively correct the model’s initial velocity estimate toward the flow-matching target Read more about it: www.alphaxiv.org/abs/2610.05538

If doomscrolling is part of your research workflow, we built something for you. Introducing Paperscrolling. Paper ideas summarized in your pocket, with key figures, results, and audio explanations! Check it out here 🚀 iOS: apps.apple.com/us/app/alpha... Android: play.google.com/store/apps/d...

🚨 [Token][ization] Paper Alert 🚨 Tokenization is a wildly understudied area of language modeling despite it having effects across all of NLP. Over the past ~8 months, 32 (!) tokenizer researchers put together the most comprehensive survey of the field. Check it out!

Linear attention is fast, but its fixed-size state limits memory This paper expands that matrix into a 3D tensor using two keys, giving an 8x larger recurrent state with only 1.2% more params At 1.3B scale, it improves long-context recall & is 5x faster at 64k context alphaxiv.org/abs/2609.36529

SFT may be better than we think. This paper rewrites expert traces into equally correct answers that look more like what the base model would naturally produce. With that, plain SFT can outperform RL while forgetting less! www.alphaxiv.org/abs/2610.02140

This new paper reuses the same Transformer blocks multiple times within each denoising step, repeatedly updating the hidden state to improve the prediction without increasing the model’s parameter count at inference-time. Read more: www.alphaxiv.org/abs/2609.40305

This paper treats an early loop from a looped transformer as a rough draft and the final loop as the refined answer It then uses the difference between them to push the final prediction further in the direction the model was already improving, with no extra training www.alphaxiv.org/abs/2610.02185

More pre-training doesn't necessarily mean better generalization for LLMs? This new paper discovers mode-hopping, where LLMs repeatedly switch between shallow pattern-matching and actual generalization, even while training loss remains stable. Read more: www.alphaxiv.org/abs/2609.33150

Most AI agents force the harness around a decoder-only Transformer But what if we change the model’s in/output shape to fit the agent? This blog suggests using recurrent memory for old history while dense attention handles recent context, reducing manual compaction alphaxiv.org/abs/2609.lan...

What if you make on-policy distillation recursive? This paper proposes that after each round, the improved model becomes both the next student and the new gold-conditioned teacher, letting better reasoning feed into future supervision. www.alphaxiv.org/abs/2609.30652

What if you could trace a model behavior back to the smallest set of internal components actually responsible for it? Matryoshka Attribution learns one ranked mask over internal components across many sparsity levels at once, letting it isolate compact causal circuits alphaxiv.org/abs/2609.25518

If world model has to reconstruct every pixel, it'll pretty much waste most of its capacity modeling irrelevant background noise. This paper found that by removing pixel decoding, it performs much better with moving distractors and natural-video backgrounds. alphaxiv.org/abs/2609.22175

Can a model learn entirely through self-play and no real data? This self-play setup can generate its own pre-training data from scratch and still teach a model general predictive structure that transfers to real-world data. www.alphaxiv.org/abs/2609.30063

This paper shows you can just use JEV for every evaluation instead of expensive LLM. JEV basically acts as a cheap first-pass judge, returning both a verdict and how confident it is. When confidence is high, keep the answer. When it’s low, escalate to a stronger LLM. alphaxiv.org/abs/2609.26550

Xiaomi dropped their new attention mechanism for their upcoming V3 model. At 1M context, it cuts prefill FLOPs by 2.92x vs HySparse and shrinks the KV cache from 6.72GB to 2.69GB, improving long-context retrieval by a lot. Read more about their new architecture here: alphaxiv.org/abs/2609.26368

VLA models are too slow for reactive robot control, so this paper lets the VLA generate actions in the background, while a lightweight RL policy uses the latest observation to edit and select actions in real time This gives from 42% to 97% with just 10min of online data alphaxiv.org/abs/2609.18207

Xiaomi MiMo just completed their largest RL scaling run so far, openly. With the run costing $2.6M for Pro and $0.9M for Flash, with ~44% spent on rollouts, 41-44% on training, and 14% on grading. Read more about it: www.alphaxiv.org/pdf/2609.mim...

This paper proposes one world-modeling framework that works across multiple fields Through Orthogonal Predictive Factorization, it splits a single JEPA latent state into complementary factors that predict different parts of the world and recombine into a complete state alphaxiv.org/abs/2609.20800

“In-Context Robot Learning with VLM Agents” This paper gave the frozen VLM an example of the task, even just a human video with no robot action labels, and it can translate what it sees into robot actions on the fly. alphaxiv.org/abs/2609.19138

“Dream-RSI: Recursive Self-Improvement through Evolving Worlds” This paper lets AI agents improve how they search by using past exploration as a simulator to cheaply practice better search strategies before trying them for real. www.alphaxiv.org/abs/2609.14858

“Learning to Solve Hard Problems in RL for LLMs by Never Giving Up” This paper makes RL better at hard problems by dynamically shifting sampling compute away from already-solved prompts toward problems the model still struggles to solve. Read more about it: www.alphaxiv.org/abs/2609.13443

Why Does Post-Training Quantization Work? This paper finds that pretrained LLMs naturally self-correct quantization errors across layers, while the LM head protects their most confident token predictions from the errors that remain. Read more about it: www.alphaxiv.org/abs/2609.11716

This new paper “The Last AI Built by Humans” proposed that true recursive self-improvement means AI getting better at improving itself, from choosing strategies and generating learning experiences, and eventually designing its own mechanisms. Read more about it: www.alphaxiv.org/abs/2609.11873

“Thinking with Looped Flows” trains recurrent reasoning as iterative denoising, so each loop turns a noisy guess into a better solution while carrying state forward. This makes extra test-time compute more reliable, reaching 58.8% on ARC-AGI-1 and 12.2% on ARC-AGI-2 www.alphaxiv.org/pdf/2609.11801

DeepSeek-V4.1-Flash DeepSeek's latest model introduces a new Causal Encoder-Decoder and CSA2 architecture that nearly halves prefill compute, compresses KV across layers, and reduces persistent KV storage by ~8x while still supporting 1M-token contexts. www.alphaxiv.org/abs/2609.dee...

This proof for one of the seven Millennium Prize Problems was generated by an unreleased OpenAI model beyond GPT-6 Astra. It basically answered a core question in fluid dynamics whether or not a perfectly smooth 3D flow can develop infinite velocity in finite time. www.alphaxiv.org/pdf/2609.nav...

On-Policy Distillation is surprisingly data-efficient, with a single query recovering 72% of the full-data gain because rollouts already cover 71.5% of the relevant state space. So the bottleneck is how slowly the student absorbs dense token-level teacher supervision www.alphaxiv.org/abs/2609.04172

“Flow Reasoning Models” This paper trains flow models to iteratively correct their own intermediate mistakes, reaching 99.5% on Sudoku-Extreme while matching the next-best method with 44x fewer inference FLOPs. www.alphaxiv.org/abs/2606.29150