nr-iqa-eval
A Lightweight Multi-Metric No-Reference Image Quality Assessment Framework for UAV Imaging — Aglin et al. (2026) (arXiv:2604.13112, 2026)
What this evaluates
This benchmark evaluates no-reference image quality assessment (NR-IQA) models on their ability to predict human-perceived image quality without a pristine reference. It probes how well a model captures diverse authentic and synthetic distortions (e.g., blur, noise, exposure, haze) and maintains monotonic and linear correlation with crowd-sourced Mean Opinion Scores (MOS).
Datasets
- KonIQ-10k — total 10073; splits: test (10073)
- LIVE Challenge — total 1162; splits: test (1162)
- KADID-10k — total 10125; splits: test (10125)
- TID2013 — total 3000; splits: test (3000)
- BIQ2021 — total 12000; splits: test (12000)
- IP102-IQA — total 35000; splits: test (35000)
Metrics
SRCC (primary) — range: [-1, 1]
- Spearman rank correlation coefficient measuring monotonicity between predicted scores and human MOS labels. Computed as 1 - (6 * sum(d_i^2)) / (n * (n^2 - 1)), where d_i is the rank difference between prediction and label.
PLCC (primary) — range: [-1, 1]
- Pearson linear correlation coefficient measuring linear agreement after predictions are mapped to the MOS scale. Requires fitting a standard five-parameter logistic mapping on a held-out set to correct scale and bias before computing correlation.
Accuracy / Precision / Recall / F1 — range: [0, 1]
- Classification metrics used for diagnostic validation on IP102-IQA. Accuracy is the fraction of correctly classified images. Precision, Recall, and F1 are computed per distortion class, with macro-averaged and weighted F1 variants to handle class imbalance.
Input / output format
Input: Single RGB image (no reference image provided).
Output: A single continuous quality score representing predicted perceptual quality.
Scoring recipe
def compute_iqa_metrics(predictions, mos_labels, heldout_indices):
# SRCC: Spearman rank correlation
srcc = spearman_rank_correlation(predictions, mos_labels)
# PLCC: Pearson correlation after 5-param logistic mapping
heldout_preds = predictions[heldout_indices]
heldout_mos = mos_labels[heldout_indices]
logistic_params = fit_5param_logistic(heldout_preds, heldout_mos)
mapped_preds = apply_logistic_mapping(predictions, logistic_params)
plcc = pearson_correlation(mapped_preds, mos_labels)
# Bootstrap resampling for 95% CI (100 iterations)
bootstrap_means = []
for _ in range(100):
sample_idx = np.random.choice(len(predictions), replace=True)
bootstrap_means.append(spearman_rank_correlation(predictions[sample_idx], mos_labels[sample_idx]))
return srcc, plcc, np.mean(bootstrap_means)
Common pitfalls
- PLCC must be computed after fitting a standard five-parameter logistic mapping on a held-out set; using a simple linear fit or skipping the held-out split will yield incorrect values.
- Reported SRCC/PLCC medians across standardized train/test partitions are used to reduce split randomness; evaluating on a single arbitrary split may produce non-comparable results.
- IP102-IQA lacks subjective MOS labels; its classification metrics are strictly for diagnostic validation of distortion cues, not for final quality ranking.
Evidence (verbatim from paper)
For the five public datasets, MM-IQA was evaluated using standard NR-IQA metrics. SRCC was computed between the model predictions and the provided subjective scores to assess monotonicity, and PLCC was computed after fitting a standard five-parameter logistic mapping on a held-out set to correct scale and bias before accuracy reporting.
Citation
@misc{aglin2026mmiqa,
title={A Lightweight Multi-Metric No-Reference Image Quality Assessment Framework for UAV Imaging},
author={Aglin et al. (2026)},
year={2026},
note={arXiv:2604.13112}
}
1---2name: nr-iqa-eval3description: This benchmark evaluates no-reference image quality assessment (NR-IQA) models on their ability to predict human-perceived image quality without a pristine reference. It probes how well a model captures diverse authentic and synthetic distortions (e.g., blur, noise, exposure, haze) and maintains monotonic and linear correlation with crowd-sourced Mean Opinion Scores (MOS). Use when the user wants to benchmark on KonIQ-10k, LIVE Challenge, KADID-10k, TID2013, BIQ2021, IP102-IQA, or asks about evaluating this task. Reports SRCC, PLCC.4metadata:5 source: https://skillmd.com/skills/qhjqhj00/nr-iqa-eval6 skill_kind: dataset_eval7 source_arxiv: 2604.131128 bibtex_key: aglin2026mmiqa9 confidence: high10---1112# nr-iqa-eval1314> A Lightweight Multi-Metric No-Reference Image Quality Assessment Framework for UAV Imaging — Aglin et al. (2026) (arXiv:2604.13112, 2026)1516## What this evaluates1718This benchmark evaluates no-reference image quality assessment (NR-IQA) models on their ability to predict human-perceived image quality without a pristine reference. It probes how well a model captures diverse authentic and synthetic distortions (e.g., blur, noise, exposure, haze) and maintains monotonic and linear correlation with crowd-sourced Mean Opinion Scores (MOS).1920## Datasets2122- **KonIQ-10k** — total 10073; splits: test (10073)23- **LIVE Challenge** — total 1162; splits: test (1162)24- **KADID-10k** — total 10125; splits: test (10125)25- **TID2013** — total 3000; splits: test (3000)26- **BIQ2021** — total 12000; splits: test (12000)27- **IP102-IQA** — total 35000; splits: test (35000)2829## Metrics3031- `SRCC` **(primary)** — range: [-1, 1]32 - Spearman rank correlation coefficient measuring monotonicity between predicted scores and human MOS labels. Computed as 1 - (6 * sum(d_i^2)) / (n * (n^2 - 1)), where d_i is the rank difference between prediction and label.33- `PLCC` **(primary)** — range: [-1, 1]34 - Pearson linear correlation coefficient measuring linear agreement after predictions are mapped to the MOS scale. Requires fitting a standard five-parameter logistic mapping on a held-out set to correct scale and bias before computing correlation.35- `Accuracy / Precision / Recall / F1` — range: [0, 1]36 - Classification metrics used for diagnostic validation on IP102-IQA. Accuracy is the fraction of correctly classified images. Precision, Recall, and F1 are computed per distortion class, with macro-averaged and weighted F1 variants to handle class imbalance.3738## Input / output format3940**Input**: Single RGB image (no reference image provided).4142**Output**: A single continuous quality score representing predicted perceptual quality.4344## Scoring recipe4546```python47def compute_iqa_metrics(predictions, mos_labels, heldout_indices):48 # SRCC: Spearman rank correlation49 srcc = spearman_rank_correlation(predictions, mos_labels)50 51 # PLCC: Pearson correlation after 5-param logistic mapping52 heldout_preds = predictions[heldout_indices]53 heldout_mos = mos_labels[heldout_indices]54 logistic_params = fit_5param_logistic(heldout_preds, heldout_mos)55 mapped_preds = apply_logistic_mapping(predictions, logistic_params)56 plcc = pearson_correlation(mapped_preds, mos_labels)57 58 # Bootstrap resampling for 95% CI (100 iterations)59 bootstrap_means = []60 for _ in range(100):61 sample_idx = np.random.choice(len(predictions), replace=True)62 bootstrap_means.append(spearman_rank_correlation(predictions[sample_idx], mos_labels[sample_idx]))63 return srcc, plcc, np.mean(bootstrap_means)64```6566## Common pitfalls6768- PLCC must be computed after fitting a standard five-parameter logistic mapping on a held-out set; using a simple linear fit or skipping the held-out split will yield incorrect values.69- Reported SRCC/PLCC medians across standardized train/test partitions are used to reduce split randomness; evaluating on a single arbitrary split may produce non-comparable results.70- IP102-IQA lacks subjective MOS labels; its classification metrics are strictly for diagnostic validation of distortion cues, not for final quality ranking.7172## Evidence (verbatim from paper)7374> For the five public datasets, MM-IQA was evaluated using standard NR-IQA metrics. SRCC was computed between the model predictions and the provided subjective scores to assess monotonicity, and PLCC was computed after fitting a standard five-parameter logistic mapping on a held-out set to correct scale and bias before accuracy reporting.7576## Citation7778```bibtex79@misc{aglin2026mmiqa,80 title={A Lightweight Multi-Metric No-Reference Image Quality Assessment Framework for UAV Imaging},81 author={Aglin et al. (2026)},82 year={2026},83 note={arXiv:2604.13112}84}85```8687- arXiv: 2604.13112