Sqi Separation Margin Eval

sqi-separation-margin-eval

Task- and Metric-Specific Signal Quality Indices for Medical Time Series — Haidamous et al. (2026) (arXiv:2602.12478, 2026)

What this evaluates

Evaluates the ability of signal quality indices (SQIs) to predict downstream task performance on medical time series. It measures how well an SQI correlates with and separates high-quality from low-quality signal segments for specific tasks like R-peak detection and atrial fibrillation classification.

Datasets

• Glasgow University database (GUDb) — total ?; splits: test (-1)
• MIT-BIH Atrial Fibrillation Database (MIT-BIH AF) — total ?; splits: test (-1)
• Deepbeat test subset — total 17617; splits: test (17617)

Metrics

• optimal separation margin ($\Delta^*$) (primary) — range: other
  • The difference in expected downstream metric performance between segments classified as high quality ($q(x)=1$) and low quality ($q(x)=0$) by the SQI, optimized over a threshold $\tau$. For accuracy: $\Delta^* = \mathbb{E}[\mathbbm{1}(\hat{y}=y) \mid q(x)=1] - \mathbb{E}[\mathbbm{1}(\hat{y}=y) \mid q(x)=0]$. For F1, it uses the mean F1 score instead of the indicator function.
• Spearman correlation coefficient — range: [-1, 1]
  • Measures the monotonicity between SQI values and the mean downstream metric across 25 uniform bins over the (0,1) range.

Input / output format

Input: Segmented medical time series signals (ECG or PPG) of fixed length (10s or 30s), along with ground-truth annotations (R-peak locations or AF labels) and downstream algorithm predictions.

Output: Continuous SQI score per segment (normalized to (0,1) range), used to bin segments and compute separation margin and monotonicity.

Scoring recipe

1. Compute SQI score for each segment.
2. Bin segments into 25 uniform intervals over (0,1).
3. Calculate mean downstream metric (F1 or accuracy) per bin.
4. Compute Spearman correlation between SQI scores and bin means for monotonicity.
5. For separation margin: iterate over candidate thresholds $\tau \in [0,1]$.
6. For each $\tau$, split segments into high-quality ($q \ge \tau$) and low-quality ($q < \tau$) sets.
7. Compute $\Delta(\tau) = \text{mean}(\text{metric}_{high}) - \text{mean}(\text{metric}_{low})$.
8. Return $\Delta^* = \max_\tau \Delta(\tau)$.

Common pitfalls

• fSQI and dSQI require labeled datasets to predict ground-truth metrics, introducing a training bias not present in the unsupervised pSQI.
• SNR hyperparameters for pSQI are highly task-dependent; using default values (25dB/10dB) may underperform compared to task-specific tuning.
• The separation margin formula assumes a binary quality threshold; optimizing this threshold per dataset is required for fair comparison.

Evidence (verbatim from paper)

The optimal separation margin is $\Delta^{}=!\mathbb{E}{\mathcal{D}}!\big[\mathbbm{1}(\hat{y}!=!y)\mid q(x)!=!1\big]!-!\mathbb{E}{\mathcal{D}}!\big[\mathbbm{1}(\hat{y}!=!y)\mid q(x)!=!0\big],$ where $\hat{y}$ is the ECGFounder model’s AF prediction. ... The pSQI significantly outperforms the fSQI and dSQI with $\Delta^{}!=!0.981!-!0.612!=!0.369$ in comparison to $0.023$ and $0.086$ respectively, as seen in Table I.

Citation

@misc{haidamous2026taskmetric,
  title={Task- and Metric-Specific Signal Quality Indices for Medical Time Series},
  author={Haidamous et al. (2026)},
  year={2026},
  note={arXiv:2602.12478}
}

• arXiv: 2602.12478