The model also works with datasets containing a few hundred neurons from different animals and laboratories. There is more good stuff in the appendix of the paper and the code repository! Paper: www.biorxiv.org/content/10.1... Code and model weights: github.com/bryanlimy/Vi... 7/7

Movie-trained transformer reveals novel response properties to dynamic stimuli in mouse visual cortex

Understanding how the brain encodes complex, dynamic visual stimuli remains a fundamental challenge in neuroscience. Here, we introduce ViV1T, a transformer-based model trained on natural movies to pr...

biorxiv.org

We sincerely thank Turishcheva & Fahey et al. (2023) for organising the Sensorium challenge(s!) and for making their high-quality, large-scale mouse V1 recordings publicly available, which made this work possible! 6/7

We compared our model against SOTA models from the Sensorium 2023 challenge and showed that ViV1T is the most performant while being more computationally efficient. We also evaluated the data efficiency of the model by varying the number of training samples and neurons. 5/7

Moving beyond gratings, we used ViV1T to generate centre-surround most exciting videos (MEVs) via the Inception Loop (Walker et al. 2019). Our in vivo experiments confirmed that MEVs elicit stronger contextual modulation than gratings, natural images and videos, and most exciting images (MEIs). 4/7

ViV1T also revealed novel functional features. We found new properties of contextual responses to surround stimuli in V1 neurons, both movement- and contrast-dependent. We validated this in vivo! 3/7

ViV1T, only trained on natural movies, captured well-known direction tuning and contextual modulation of V1. Despite no built-in mechanism for modelling neuron connectivities, the model predicted feedback-dependent contextual modulation (including feedback onset delay!) (Keller et al. 2020). 2/7

We present our preprint on ViV1T, a transformer for dynamic mouse V1 response prediction. We reveal novel response properties and confirm them in vivo. With @wulfdewolf.bsky.social, Danai Katsanevaki, @arnoonken.bsky.social, @rochefortlab.bsky.social. Paper and code at the end of the thread! 🧵1/7