Tessera
Folgen

jgyou

@jgyou.bsky.social

Assistant professor of Statistics and Complex systems at UVM. Lab: https://joint-lab.github.io

49 Posts1810 Follower662 Folgt
Ted Underwood@tedunderwood.com · 12mo ago

Implementing “reasoning” as “just keep talking to yourself” feels so weird & hacky. But maybe not, according to this? They show that just-predicting-the-next-word is inherently unable to learn to solve some (permutation) problems unless you “pad” with extra thought tokens. arxiv.org/abs/2509.24125

In this technical note, we study the problem of inverse permutation learning in decoder-only transformers. Given a permutation and a string to which that permutation has been applied, the model is tasked with producing the original ("canonical") string. We argue that this task models a natural robustness property across a variety of reasoning tasks, including long-context retrieval, multiple choice QA and in-context learning. Our primary contribution is an impossibility result: we show that an arbitrary depth, decoder-only transformer cannot learn this task. This result concerns the expressive capacity of decoder-only transformer models and is agnostic to training dynamics or sample complexity. We give a pair of alternative constructions under which inverse permutation learning is feasible. The first of these highlights the fundamental role of the causal attention mask, and reveals a gap between the expressivity of encoder-decoder transformers and the more popular decoder-only architecture. The latter result is more surprising: we show that simply padding the input with "scratch tokens" yields a construction under which inverse permutation learning is possible. We conjecture that this may suggest an alternative mechanism by which chain-of-thought prompting or, more generally, intermediate "thinking" tokens can enable reasoning in large language models, even when these tokens encode no meaningful semantic information (e.g., the results of intermediate computations).
jgyou@jgyou.bsky.social · last yr.

And the code is live here: github.com/DynamicaLab/... With @allard.bsky.social and the never-online Simon Lizotte (but you can find him at siliz4.github.io)

GitHub - DynamicaLab/bigue: Bayesian inference of the hyperbolic plane embedding of complex networks.

Bayesian inference of the hyperbolic plane embedding of complex networks. - DynamicaLab/bigue

github.com

Sadamori Kojaku@skojaku.bsky.social · last yr.

A Bayesian approach to hyperbolic embedding, which can provide "error bars" for vertex angles, radial parameters, inverse temperature. Symmetry-driven embedding of networks in hyperbolic space by Simon Lizotte, Jean-Gabriel Young & Antoine Allard. www.nature.com/articles/s42...

Alessandro Vespignani@alexvespi.bsky.social · 2y ago

Wastewater from airplane toilets? We introduce a global Aircraft-Based Wastewater Surveillance Network (WWSN) for pandemic monitoring in Nature Medicine 🔗 doi.org/10.1038/s415... Aircraft-based wastewater surveillance allows for real-time, non-invasive monitoring of global pathogen spread Short 🧵

Pandemic monitoring with global aircraft-based wastewater surveillance networks - Nature Medicine

By simulating the implementation of airport-based wastewater surveillance sites at the global level, a modeling study shows how this early warning system would perform in identifying sources of pandem...

doi.org

Andrew Mercer@awmercer.bsky.social · 2y ago

We are reaching the time of year when I start hearing from undergrads who are looking for summer internships. Obviously, they should all apply to Pew, but what other research orgs have internship programs that I can tell them about? What other opportunities can I share with them?

jgyou@jgyou.bsky.social · 2y ago

I preemptively declare paper bankruptcy. Anything published on arXiv during the next 3 weeks will be permanently memory black holed.

Sam Scarpino@scarpino.bsky.social · 2y ago

1/ Influenza wastewater activity levels are currently low across the US, aside from conspicuous areas in CA where #H5N1 is spreading in dairy cows. A deeper dive into the Wastewater Scan data paints a very concerning picture and should add to our sense of urgency.

Map shows the location of wastewater sampling locations around the US. Most of the sites indicate low levels and are colored blue. However, there are a number of sites in CA with high levels and few other scattered sites around the US with medium/high levels and are colored in orange/red