Quanquan Gu
@quanquangu.bsky.social
Professor @UCLA, Research Scientist @ByteDance | Recent work: SPIN, SPPO, DPLM 1/2, GPM, MARS | Opinions are my own
Papers #2-3: arxiv.org/abs/2402.10210 and arxiv.org/abs/2405.00675 from the incredible @quanquangu.bsky.social. I really like how they explore new techniques for RLHF
Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generation
Fine-tuning Diffusion Models remains an underexplored frontier in generative artificial intelligence (GenAI), especially when compared with the remarkable progress made in fine-tuning Large Language M...
arxiv.org
Pretraining will only end once we find the optimal scaling law.
With the delivery of MARS complete, the focus now shifts to delivering new architectures.
GitHub - AGI-Arena/MARS: The official implementation of MARS: Unleashing the Power of Variance Reduction for Training Large Models
The official implementation of MARS: Unleashing the Power of Variance Reduction for Training Large Models - AGI-Arena/MARS
buff.ly
This Thanksgiving, I want to express my heartfelt gratitude to all the students, colleagues, and collaborators who have contributed to the success of SPIN, SPPO, DPLM, GPM, MARS, and many other projects. Your hard work and dedication continue to be truly inspiring.
Tulu 3 SFT mix trending on HuggingFace :D , next step make preferences and RL datasets more accessible.
OLMo 2 is out 🥳 7B and 13B trained on 5T tokens, and meticulousy instruction tuned using Tulu 3 recipe. Simply the best fully open models yet. Really proud of the work & the amazing team at @ai2.bsky.social
Replacing the mean by an arbitrary constant, part 2. This very cool fact is from Tom Cover's 1990 Shannon Lecture: #MathSky
PSGD ❤️ MARS MARS is a new exciting variance reduction technique from @quanquangu.bsky.social 's group which can help stabilize and accelerate your deep learning pipeline. All that is needed is a gradient buffer. Here MARS speeds up the convergence of PSGD ultimately leading to a better solution.
Tomorrow at 6 PM UTC, Asaf Cassel will talk about "Warm-Up Free Policy Optimization: Improved Regret in Linear Markov Decision Processes".
We are on Bluesky as well! We will keep posting on both X and here.
Just created the Starter Pack for Optimization Researchers to help you on your journey into optimization! 🚀 Did I miss anyone? Tag them or let me know what to add! go.bsky.app/VjpyyRw
Join us tomorrow for the: AI for Science summit. In person places are full, but you can still join us online (click on the below to sign up). science.ai.cam.ac.uk/events/ai-fo...
AI for Science Summit 2024
On Monday 25 and Tuesday 26 November the Accelerate Programme for Scientific Discovery are inviting members of the AI for Science community to attend an AI for Science Summit, bringing together our co...
science.ai.cam.ac.uk
In addition to the Deep Learning Theory starter pack, I've also put together a starter pack for Reinforcement Learning Theory. Let me know if you'd like to be included or suggest someone to add to the list! go.bsky.app/LWyGAAu
Graduate school application season is here again, and I just spent the whole weekend writing a few letters! 🎓 When submitting, many schools still require overly detailed forms filled with endless questions and checkboxes. Isn’t it time to simplify this process and focus on what truly matters?
A gentle reminder to @iclr-conf.bsky.social authors: your revision must not exceed 10 pages, adhering to the same requirements as the original submission.
Just put together a starter pack for Deep Learning Theory. Let me know if you'd like to be included or suggest someone to add to the list! go.bsky.app/2qnppia
RLC will be held at the Univ. of Alberta, Edmonton, in 2025. I'm happy to say that we now have the conference's website out: rl-conference.cc/index.html Looking forward to seeing you all there! @rl-conference.bsky.social #reinforcementlearning
It's Sunday morning so taking a minute for a nerdy thread (on math, tokenizers and LLMs) of the work of our intern Garreth By adding a few lines of code to the base Llama 3 tokenizer, he got a free boost in arithmetic performance 😮 [thread]
Write math on 🦋 with UnicodeIt! For example: θ ∈ ℝⁿ or pp̅ → μ⁺μ⁻ Use website or install system-wide in Linux, macOS, or windows www.unicodeit.net (Created several years ago with @svenkreiss.bsky.social)
I’m increasingly unsure there are specific rules/laws for pretraining schedule except that peak LR instability means that you need a warmup and a cooldown. And in the post-Chinchilla era you probably want to slow cook the model (so go easy on lr, smooth the schedule).
Everything you want to know about FSDP and more... FSDP works on layers at a time and enables perf wins by overlapping the data-fetching part of the next layer with the computation of the current layer. This overlap means that the data-fetching can be “hidden”.
LLMs generate novel word sequences not contained in their pretraining data. However, compared to humans, models generate significantly fewer novel n-grams. RLHF = 30% *more* copying than base! Awesome work from the awesome Ximing Lu (gloriaximinglu.github.io) et al. 🤩 arxiv.org/pdf/2410.04265
🚀 Excited to announce KVPress — our open-source library for efficient LLM KV cache compression! 👉 Check it out (and drop a ⭐): github.com/NVIDIA/kvpress 🔗 Full details in the thread 🧵 (1/4)
if you have a @github.com profile, can i ask you to update it with your @bsky.app handle? 🙏 👉 it enables some very cool integrations, like auto curated feeds and starter packs for contributors and tech
What are some "nice-in-theory but unhelpful-in-practice" approaches in LLMs?
