New preprint: NPI and depth charge illusions in Pythia LLMs of different sizes and at different training checkpoints. NPI illusion goes away but depth charge illusion stays or even gets stronger. Amazingly for me, I come out in favor of... construction grammar? arxiv.org/abs/2603.27855
What can LLMs tell us about the mechanisms behind polarity illusions in humans? Experiments across model scales and training steps
I use the Pythia scaling suite (Biderman et al. 2023) to investigate if and how two well-known polarity illusions, the NPI illusion and the depth charge illusion, arise in LLMs. The NPI illusion becom...
arxiv.org