Nora Belrose

@norabelrose.bsky.social

AI, philosophy, spirituality Head of interpretability research at EleutherAI, but posts are my own views, not Eleuther’s.

why don't more people become zoroastrian? it's where judaism and christianity got the idea of ethical monotheism, afterlife, and final judgment but without any of their baggage (no eternal hell, no historically questionable dogmas, etc.)

If we care only about appearances, outcomes, and results then AI will outcompete humans at everything If we care about the process used to create things then humans can still have jobs and meaningful lives The idea that ends can be detached from means is the root of many evils

data attribution is a special case of data causality: estimating the causal effect of either learning or unlearning one datapoint (or set of datapoints) on the neural network's behavior on other datapoints

Neural networks don't have organs. They aren't made of fixed mechanisms. They have flows of information and intensities of neural activity. They can't be organized into a set of parts with fixed functions. In the words of Gilles Deleuze, they're bodies without organs (BwO).

This seems like a cool way to use an adaptive amount of compute per token. I speculate that models like these will have more faithful CoT since they don't get to do "extra" reasoning on easy tokens https://arxiv.org/abs/2404.02258

Mixture-of-Depths: Dynamically allocating compute in...

Transformer-based language models spread FLOPs uniformly across input sequences. In this work we demonstrate that transformers can instead learn to dynamically allocate FLOPs (or compute) to...

arxiv.org

Pro tip: if you want to implement TopK SAEs efficiently, and don't want to deal with Triton, just use this function for the decoder, it's much faster than the naive dense matmul implementation https://pytorch.org/docs/stable/generated/torch.nn.functional.embedding_bag.html

Evolutionary biology can learn things from machine learning. Natural selection alone doesn't explain "train-test" or "sim-to-real" generalization, which clearly happens. At every level of organization, life can zero-shot adapt to novel situations. https://www.youtube.com/watch?v=jJ9O5H2AlWg

Truth is relative, when it comes to the physical state of the universe. But we should accept the existence of perspective-neutral facts about how perspectives relate to one another, to avoid vicious skeptical paradoxes. https://arxiv.org/abs/2410.13819

How do a neural network's final parameters depend on its initial ones? In this new paper, we answer this question by analyzing the training Jacobian, the matrix of derivatives of the final parameters with respect to the initial parameters. https://arxiv.org/abs/2412.07003

Bild

Bombshell new paper on the simulation argument, multiverses, and cosmological fine-tuning: "...self-locating credences are ‘subjective’ in the sense that they are not rationally constrained by anything at all, except possibly the requirement of probabilistic consistency." arxiv.org/abs/2409.05259

Against Self-Location

I distinguish between pure self-locating credences and superficially self-locating credences, and argue that there is never any rationally compelling way to assign pure self-locating credences. I firs...

arxiv.org