We present The Transformer Cookbook: a collection of recipes for programming algorithms directly into transformers! Hungry for an induction head? Craving a Dyck language recognizer? We show you step-by-step how to cook up transformers for these algorithms and many more!

The Transformer Cookbook

We present the transformer cookbook: a collection of techniques for directly encoding algorithms into a transformer's parameters. This work addresses the steep learning curve of such endeavors, a prob...

arxiv.org

Excited to head to @NeurIPSConf today! I'll be presenting our work on the representational capabilities of Transformers and RNNs/SSMs. If you're interested in meeting up to discuss research or chat, feel free to reach out via DM or email!

I wanted my Algorithms students to program NP hardness reductions so we developed Karp. A domain specific language for writing Karp reductions. Our students are quite good with a debugger, so reducing learning Theory to debugging seemed like a win. docs.racket-lang.org/karp/index.h...