SPARKS encodes the spikes of neurons in the somatosensory cortex of a monkey performing a centre-out reaching task into two-dimensional latent embeddings, from which it decodes the position of the hand (trial-averaged test data, Chowdhury et al., 2020).
Understanding how the brain represents sensory information and triggers behavioural responses is a fundamental goal in neuroscience. Recent advances in neuronal recording techniques aim to progress towards this milestone, yet the resulting high-dimensional responses are challenging to interpret and link to relevant variables. Although existing machine learning models propose to do so, they often sacrifice interpretability for predictive power, effectively operating as black boxes. In this work, we introduce SPARKS, a biologically inspired model capable of high decoding accuracy and interpretable discovery within a single framework. SPARKS adapts the self-attention mechanism of large language models to extract information from the timing of single spikes and the sequence in which neurons fire using Hebbian learning. Trained with a criterion inspired by predictive coding to enforce temporal coherence, our model produces low-dimensional latent embeddings that are robust across sessions and animals. By directly capturing the underlying data distribution through a generative encoding-decoding framework, SPARKS matches or exceeds state-of-the-art decoders across diverse electrophysiology and calcium imaging datasets from the motor, visual and entorhinal cortices. Crucially, the Hebbian coefficients learned by the model are interpretable, allowing us to infer directed functional interactions and recover the known functional hierarchy of the mouse visual cortex. Overall, SPARKS unifies representation learning, high-performance decoding and model interpretability in a single framework by bridging neuroscience and AI, providing a powerful and versatile tool for dissecting neural computations and marking a step towards the next generation of biologically inspired intelligent systems.
SPARKS (Sequential Predictive Autoencoder for the Representation of spiKing Signals) is a biologically inspired model for the analysis of neural recordings. Machine learning models of neural data often sacrifice interpretability for predictive power. SPARKS instead unifies high-performance decoding, representation learning and interpretability in a single framework.
It combines a variational autoencoder with a Transformer whose attention mechanism follows Hebbian learning, operating directly on spike trains or calcium imaging data with minimal preprocessing. SPARKS produces low-dimensional latent embeddings that are dynamically faithful to the underlying signals, and robust across repetitions, sessions and animals. It can be trained with labels, to decode behaviour or stimuli, or without, to explore the structure of the data. Finally, the coefficients computed by its Hebbian attention layer reflect directed functional interactions between neurons, giving interpretable insights into the organisation of neural circuits.
Information in neural recordings is carried by the relative timing and order of sparse, asynchronous spikes, whereas the dot-product attention of Transformers compares tokens presented simultaneously, and relies on positional embeddings to recover their order. At the heart of SPARKS, the Hebbian attention layer fuses these two steps: the attention coefficient between each pair of neurons is computed at every time step by spike-timing-dependent plasticity (STDP). Eligibility traces are potentiated when neurons spike and decay otherwise, so that a coefficient grows when one neuron fires shortly before the other. Hebbian attention is therefore a spatiotemporal operator, sensitive both to which neurons fire and to the order in which they fire, at single-spike resolution.
The time constants and update magnitudes of the traces can be kept fixed (SPARKS-fixed), or learned for each pair of neurons (SPARKS-learned), which benefits large datasets. Several heads with different time constants can capture interactions at several timescales. The resulting neuron embeddings are then compressed by a Perceiver-style bottleneck into a representation whose size does not depend on the number of neurons. A single model can thus be trained on several recordings with different neuron counts, with a Hebbian layer specific to each session, and all other parameters shared.
SPARKS is trained with a variational objective based on a predictive causally conditioned distribution, inspired by predictive coding: at every time step, a window of past latent embeddings is used to predict several future samples of a reference signal. The reference signal can be the recording itself (unsupervised learning), or behavioural and sensory variables (supervised learning). By forcing the model to predict the future evolution of the signal, this objective introduces temporal coherence in the latent embeddings. The latents are regularised towards a learned autoregressive prior, and long recordings are handled with truncated backpropagation through time.
Because the session-specific Hebbian layers map every recording to a shared space, a trained model can analyse new sessions by adding a new Hebbian layer and fine-tuning: in our experiments, a new session reached its final accuracy within about 50 epochs, without loss of accuracy on the sessions the model was trained on.
We showcase SPARKS on electrophysiology and calcium imaging datasets from the motor, visual and entorhinal cortices:
Try it on your own data with the documentation and the demo notebooks.
@article{skatchkovsky2024sparks,
title = {A Biologically Inspired Attention Model for Neural Signal Analysis},
author = {Skatchkovsky, Nicolas and Glazman, Natalia and Egea-Weiss, Alexander and Sadeh, Sadra and Iacaruso, Florencia},
journal = {bioRxiv},
year = {2024},
doi = {10.1101/2024.08.13.607787},
url = {https://www.biorxiv.org/content/10.1101/2024.08.13.607787}
}