Dynamical Phases of Short-Term Memory Mechanisms in RNNs

Kurtkaya, B., Dinc, F., Yuksekgonul, M., Blanco-Pozo, M., Cirakman, E., Schnitzer, M., Yemez, Y., Tanaka, H., Yuan, P., & Miolane, N.

We uncover two distinct mechanisms—slow-point manifolds and limit cycles—that support short-term memory in recurrent networks, derive theoretical scaling laws governing their emergence, and validate these findings across 35,000 RNNs, offering new insights and testable predictions for neuroscience.