Liam Storan, Nikos Karantzas, Klindt, D., Willeke, K. F., Deng, D., Michał Gerasimiuk, Xaq Pitkow, Franke, K., Miolane, N., Sanborn, S., & Tolias, A. S.
Abstract
How does the brain transform the complex visual world into meaningful representations that facilitate generalization across diverse conditions? One hypothesis is that the geometric structure of neural manifolds mirrors causal structures in the environment, facilitating strong generalization across natural contexts. The analysis of neural manifold structure has yielded neuroscientific insights in domains such as navigation and motor control, which often possess simple, low-dimensional structure. Vision, however, presents unique challenges due to its more complex, high-dimensional, hierarchical structure. Leveraging a digital twin model of primate V4 neurons, we conduct targeted in silico experiments that allow us to systematically investigate the relationship between the structure of the visual world and its encoding in the visual cortex. Our findings reveal structural equivalences between world properties and neural activity patterns for rotating objects and textures. Specifically, we demonstrate the emergence of equivariant representations that disentangle latent factors such as object identity and orientation. Finally, we demonstrate that these representations facilitate out-of-distribution generalization, as a decoder trained to linearly decode the orientation of one texture can successfully transfer to novel textures. Remarkably, artificial neural networks trained on object recognition tasks exhibit similar geometric principles. These results provide empirical support for the mirroring hypothesis in visual processing and suggest universal principles govern the formation of neural representations across biological and artificial vision.
Citation
Liam Storan, Nikos Karantzas, Klindt, D., Willeke, K. F., Deng, D., Michał Gerasimiuk, Xaq Pitkow, Franke, K., Miolane, N., Sanborn, S., & Tolias, A. S. (2024). Emergent Geometry in Neural Representations of the Visual World.
BibTeX
@misc{storan2024emergent, title={Emergent Geometry in Neural Representations of the Visual World}, author={Liam Storan and Nikos Karantzas and David Klindt and Konstantin Friedrich Willeke and Dongrui Deng and Micha{\l} Gerasimiuk and Xaq Pitkow and Katrin Franke and Nina Miolane and Sophia Sanborn and Andreas S. Tolias}, year={2024} }
