Our new work on scaling laws that includes compute, model size, and number of samples. The analysis involves an extremely fine-grained analysis of online sgd built up over the last 8 years of understanding sgd on simple toy models (tensors, single index models, multi index model)
Eshaan Nichani@eshaannichani.bsky.social · last yr.Excited to announce a new paper with Yunwei Ren, Denny Wu, @jasondeanlee.bsky.social! We prove a neural scaling law in the SGD learning of extensive width two-layer neural networks. arxiv.org/abs/2504.19983 🧵below (1/10)
