Nr Iqa Eval

nr-iqa-eval

A Lightweight Multi-Metric No-Reference Image Quality Assessment Framework for UAV Imaging — Aglin et al. (2026) (arXiv:2604.13112, 2026)

What this evaluates

This benchmark evaluates no-reference image quality assessment (NR-IQA) models on their ability to predict human-perceived image quality without a pristine reference. It probes how well a model captures diverse authentic and synthetic distortions (e.g., blur, noise, exposure, haze) and maintains monotonic and linear correlation with crowd-sourced Mean Opinion Scores (MOS).

Datasets

• KonIQ-10k — total 10073; splits: test (10073)
• LIVE Challenge — total 1162; splits: test (1162)
• KADID-10k — total 10125; splits: test (10125)
• TID2013 — total 3000; splits: test (3000)
• BIQ2021 — total 12000; splits: test (12000)
• IP102-IQA — total 35000; splits: test (35000)

Metrics

• SRCC (primary) — range: [-1, 1]
  • Spearman rank correlation coefficient measuring monotonicity between predicted scores and human MOS labels. Computed as 1 - (6 * sum(d_i^2)) / (n * (n^2 - 1)), where d_i is the rank difference between prediction and label.
• PLCC (primary) — range: [-1, 1]
  • Pearson linear correlation coefficient measuring linear agreement after predictions are mapped to the MOS scale. Requires fitting a standard five-parameter logistic mapping on a held-out set to correct scale and bias before computing correlation.
• Accuracy / Precision / Recall / F1 — range: [0, 1]
  • Classification metrics used for diagnostic validation on IP102-IQA. Accuracy is the fraction of correctly classified images. Precision, Recall, and F1 are computed per distortion class, with macro-averaged and weighted F1 variants to handle class imbalance.

Input / output format

Input: Single RGB image (no reference image provided).

Output: A single continuous quality score representing predicted perceptual quality.

Scoring recipe

def compute_iqa_metrics(predictions, mos_labels, heldout_indices):
    # SRCC: Spearman rank correlation
    srcc = spearman_rank_correlation(predictions, mos_labels)
    
    # PLCC: Pearson correlation after 5-param logistic mapping
    heldout_preds = predictions[heldout_indices]
    heldout_mos = mos_labels[heldout_indices]
    logistic_params = fit_5param_logistic(heldout_preds, heldout_mos)
    mapped_preds = apply_logistic_mapping(predictions, logistic_params)
    plcc = pearson_correlation(mapped_preds, mos_labels)
    
    # Bootstrap resampling for 95% CI (100 iterations)
    bootstrap_means = []
    for _ in range(100):
        sample_idx = np.random.choice(len(predictions), replace=True)
        bootstrap_means.append(spearman_rank_correlation(predictions[sample_idx], mos_labels[sample_idx]))
    return srcc, plcc, np.mean(bootstrap_means)

Common pitfalls

• PLCC must be computed after fitting a standard five-parameter logistic mapping on a held-out set; using a simple linear fit or skipping the held-out split will yield incorrect values.
• Reported SRCC/PLCC medians across standardized train/test partitions are used to reduce split randomness; evaluating on a single arbitrary split may produce non-comparable results.
• IP102-IQA lacks subjective MOS labels; its classification metrics are strictly for diagnostic validation of distortion cues, not for final quality ranking.

Evidence (verbatim from paper)

For the five public datasets, MM-IQA was evaluated using standard NR-IQA metrics. SRCC was computed between the model predictions and the provided subjective scores to assess monotonicity, and PLCC was computed after fitting a standard five-parameter logistic mapping on a held-out set to correct scale and bias before accuracy reporting.

Citation

@misc{aglin2026mmiqa,
  title={A Lightweight Multi-Metric No-Reference Image Quality Assessment Framework for UAV Imaging},
  author={Aglin et al. (2026)},
  year={2026},
  note={arXiv:2604.13112}
}

• arXiv: 2604.13112