qhjqhj00 sensorium-2023-eval
Retrospective for the Dynamic Sensorium Competition for predicting large-scale mouse primary visual cortex activity from videos — Turishcheva et al. (2024) (arXiv:2407.09100, 2024)
What this evaluates
Predicts single-neuron responses in mouse primary visual cortex from dynamic video stimuli and behavioral covariates, probing spatio-temporal neural decoding and out-of-distribution generalization.
Datasets
- SENSORIUM 2023 — total ?; splits: train (-1), test (-1), ood (-1); repo https://github.com/eyer-lab/sensorium_2023
Metrics
R^2(primary) — range: [0, 1]- Coefficient of determination measuring the proportion of variance in the actual neural responses explained by the model predictions.
single-trial correlation (ρ_st)— range: [-1, 1]- Pearson correlation coefficient computed on single-trial neural responses.
average correlation (ρ_ta)— range: [-1, 1]- Pearson correlation coefficient computed on temporally averaged neural responses.
Input / output format
Input: Spatio-temporal video frames (processed in 30-frame batches) and behavioral covariates (e.g., pupil size, mouse identity).
Output: Predicted neural response values (e.g., spike counts or firing rates) for each recorded neuron.
Scoring recipe
def compute_r2(predictions, gold):
ss_res = np.sum((gold - predictions) ** 2)
ss_tot = np.sum((gold - np.mean(gold)) ** 2)
return 1 - (ss_res / ss_tot)
# Report Pearson correlation for ρ_st and ρ_ta on single-trial and averaged responses respectively.
Common pitfalls
- Using subject-specific readout modules increases training difficulty and may weaken the shared core module's performance.
- Out-of-distribution generalization to novel stimuli requires architectures that explicitly capture temporal dynamics rather than relying on static spatial features.
- Top performance relies on ensemble strategies across multiple random seeds rather than single-model runs.
Evidence (verbatim from paper)
On the live-test evaluation, the improvement of the core module, replacing 3D convolution with 3D swim transformer, resulted in an R^2 improvement of 0.045 (from 0.188 to 0.233). Enhancements in the readout module, replacing Gaussian readout to Hierarchical Gaussian readout, further improved the model by 0.018 (from 0.233 to 0.251). The final ensemble strategy yielded an overall prediction score of 0.276.
Citation
@misc{turishcheva2024sensorium,
title={Retrospective for the Dynamic Sensorium Competition for predicting large-scale mouse primary visual cortex activity from videos},
author={Turishcheva et al. (2024)},
year={2024},
note={arXiv:2407.09100}
}
- arXiv: 2407.09100