This is fascinating. Very large models can read the weights of small models and intuit what those models were trained to do, and make direct weight changes to improve them. rikiyat.github.io/blog/weight-...
Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights
We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights, well past what we’d expect from humans or conventional algorithms.
rikiyat.github.io