Understanding and Controlling the Geometry of Memory Organization in RNNs

Haputhanthri, U., Storan, L., Jiang, Y., Raheja, T., Shai, A., Akengin, O., Miolane, N., Schnitzer, M. J., Dinc, F., & Tanaka, H

We investigate abrupt learning in RNNs trained on short-term memory tasks, showing that geometric restructuring in phase space precedes sudden performance gains, and introduce a temporal consistency regularization that promotes efficient, biologically plausible learning and attractor formation.