With other folks at 🍏, @brunokm.bsky.social has worked on a complete(d) parameterisation for NNs that can *transfer* locally tuned hyperparameters: tune optimizers' parameters (e.g. LR) *per module/depth* using an evolutionary search on small models → they transfer perf. gains to much larger models

Bruno Mlodozeniec@brunokm.bsky.social · 7mo ago

In our new work — Complete(d)P — we try to answer 3 questions about hyperparameter (HP) scaling: ● How to transfer across model size, tokens&batch-size?→ Complete(d)P ● Do per-module HPs matter? ✔️2x speed-ups possible ● Do they transfer to larger scale? ✔️ With the right parameterisation

In our new work — Complete(d)P — we try to answer 3 questions about hyperparameter (HP) scaling: ● How to transfer across model size, tokens&batch-size?→ Complete(d)P ● Do per-module HPs matter? ✔️2x speed-ups possible ● Do they transfer to larger scale? ✔️ With the right parameterisation

It's that time of the year! 🎁 The Apple Machine Learning Research (MLR) team in Paris is hiring a few interns, to do cool research for ±6 months 🚀🚀 & work towards publications/OSS. Check requirements and apply: ➡️ jobs.apple.com/en-us/detail... More❓→ ✉️ mlr_paris_internships@group.apple.com

How do you identify training data responsible for an image generated by your diffusion model? How could you quantify how much copyrighted works influenced the image? In our ICLR oral paper we propose how to approach such questions scalably with influence functions.

Rich Turner with other members of our group recently published a paper on Aardvark — end-to-end weather prediction with deep learning — in Nature, and it was just featured in The Guardian and Financial Times! www.theguardian.com/technology/2...

AI-driven weather prediction breakthrough reported

Researchers say Aardvark Weather uses thousands of times less computing power and is much faster than current systems

theguardian.com

Myself, James and and Shreyas will be at NeurIPS presenting this work. Come chat to us if you’re interested!

James Allingham@jamesallingham.bsky.social · 2y ago

I'll be at NeurIPS next week, presenting our work "A Generative Model of Symmetry Transformations." In it, we propose a symmetry-aware generative model that discovers which (approximate) symmetries are present in a dataset and can be leveraged to improve data efficiency. 🧵⬇️

Diffusion models are so ubiquitous, but it's difficult to find an introduction that is concise, simple and comprehensive. My supervisor Rich Turner (with me & some other students) has written an introduction to diffusion models that fills this gap: arxiv.org/abs/2402.04384

Denoising Diffusion Probabilistic Models in Six Simple Steps

Denoising Diffusion Probabilistic Models (DDPMs) are a very popular class of deep generative model that have been successfully applied to a diverse range of problems including image and video generati...

arxiv.org