Omnispectra Eval

omnispectra-eval

OmniSpectra: A Unified Foundation Model for Native Resolution Astronomical Spectra — Islam et al. (2026) (arXiv:2601.15351, 2026)

What this evaluates

Evaluates a foundation model's ability to process native-resolution astronomical spectra of variable lengths without resampling, and assesses its zero-shot, few-shot, and supervised performance on stellar property estimation and source classification tasks across diverse spectroscopic surveys.

Datasets

• OmniSpectra Multi-Survey Corpus — total 5543000; splits: train (5488000), val (55000)

Metrics

• Mean-Squared Error (primary) — range: other
  • Standard mean squared error calculated as the average of the squared differences between predicted and true property values: MSE = (1/N) * Σ(y_true - y_pred)^2. Lower values indicate better performance. The paper reports a weighted average across multiple stellar targets.

Input / output format

Input: Variable-length astronomical spectra provided as HDF5 files, tokenized via overlapping patches of 20 flux points (10-point overlap), with each instance independently normalized. Wavelengths are encoded sinusoidally.

Output: Continuous numerical values for stellar/galactic properties (e.g., temperature, metallicity) or class labels for source classification, depending on the downstream task.

Scoring recipe

def compute_mse(y_true, y_pred, weights=None):
    squared_diffs = (y_true - y_pred) ** 2
    if weights is not None:
        mse = np.average(squared_diffs, weights=weights)
    else:
        mse = np.mean(squared_diffs)
    return mse

Common pitfalls

• Downstream evaluation splits are randomly sampled (20k examples, 50/50 train-test) per dataset rather than using standard held-out survey splits, which may not reflect real-world domain shift.
• The validation set (55k spectra) is a fixed 1% random subsample of the pretraining corpus, not a separate downstream validation set.
• Variable-length inputs require adaptive patching; models that force fixed-length resampling will lose native resolution fidelity.

Evidence (verbatim from paper)

Table 3: Mean-Squared Error (↓) of star property estimation on the SDSS dataset and its subsets. We report the weighted average across the targets. The best and second-best results are in bold and underline. OmniSpectra significantly outperforms other foundation models and even custom supervised models with little tuning.

Citation

@misc{islam2026omnispectra,
  title={OmniSpectra: A Unified Foundation Model for Native Resolution Astronomical Spectra},
  author={Islam et al. (2026)},
  year={2026},
  note={arXiv:2601.15351}
}

• arXiv: 2601.15351