Identifying Interpretable Visual Features in Artificial and Biological Neural Systems

Klindt, D., Sanborn, S., Acosta, F., Poitevin, F., Miolane, N.

Many neurons in deep networks exhibit mixed selectivity, representing multiple unrelated features, which supports the hypothesis that features are represented in superposition rather than by individual neurons. We introduce an automated method to quantify visual interpretability and identify meaningful directions in activation space that are more intuitive than single neuron responses. Applying this to both artificial and biological neural data, we find evidence that superposition may underlie robust and efficient representations in the brain as well.