SPARKS: A Biologically Inspired Neural Attention Model for the Analysis of Sequential Spiking Patterns

Understanding how the brain represents sensory information and triggers behavioural responses is a fundamental goal in neuroscience. Despite advances in neuronal recording techniques, linking the resulting high dimensional responses to relevant variables remains challenging. Inspired by recent progress in machine learning, we propose a novel self-attention mechanism that generates reliable latent representations by sequentially extracting information from the precise timing of single spikes through Hebbian learning. We train a variational autoencoder encompassing the proposed attention layer using an information-theoretic criterion inspired by predictive coding to enforce temporal coherence in the latent representations. The resulting model, SPARKS, produces interpretable embeddings from just tens of neurons, demonstrating robustness across animals and sessions. Through unsupervised and supervised learning, SPARKS generates meaningful low-dimensional representations of high-dimensional recordings and offers state-of-the-art prediction capabilities for behavioural variables on diverse electrophysiology and calcium imaging datasets. Notably, we capture oscillatory sequences from the medial entorhinal cortex (MEC) at unprecedented precision, compare latent representation of natural scenes across sessions and animals, and reveal the hierarchical organisation of the mouse visual cortex from simple datasets. Combining machine learning models with biologically inspired mechanisms, SPARKS provides a promising solution for revealing large-scale network dynamics. Its capacity to generalize across animals and behavioural states suggests SPARKS potential to estimate the animal’s latent generative model of the world.