Hierarchical Mlc Image Eval

hierarchical-mlc-image-eval

HELM: Hierarchical and Explicit Label Modeling with Graph Learning for Multi-Label Image Classification — Stoimchev et al. (2026) (arXiv:2603.11783, 2026)

What this evaluates

Evaluates a model's ability to perform hierarchical multi-label image classification on remote sensing scenes. It probes how well the model captures label dependencies and hierarchy structures while predicting multiple overlapping categories per image.

Datasets

• UCM — total ?; splits: test (-1)
• AID — total ?; splits: test (-1)
• DFC-15 — total ?; splits: test (-1)
• MLRSNet — total ?; splits: test (-1)

Metrics

• AUPRC (primary) — range: [0, 1]
  • Area under the precision-recall curve computed across all classes, typically averaged macro to treat all labels equally regardless of frequency.
• Ranking Loss — range: [0, 1]
  • Measures the average fraction of label pairs that are incorrectly ordered, penalizing cases where a relevant label is ranked lower than a non-relevant one.

Input / output format

Input: Remote sensing scene images.

Output: Binary or probability scores for each hierarchical label in the predefined taxonomy.

Scoring recipe

def compute_metrics(y_true, y_pred_scores):
    # y_true: binary matrix (n_samples, n_labels)
    # y_pred_scores: float matrix (n_samples, n_labels)
    auprc = average_precision_score(y_true, y_pred_scores, average='macro')
    ranking_loss = ranking_loss_score(y_true, y_pred_scores)
    return {'AUPRC': auprc, 'Ranking Loss': ranking_loss}

Common pitfalls

• Ranking Loss is often confused with standard ranking metrics like NDCG; it specifically measures pairwise label order violations in multi-label settings.
• AUPRC should be averaged macro across all hierarchical levels/classes, not micro, to avoid bias toward frequent labels.
• Hierarchical constraints (parent-child label dependencies) are not enforced during inference in the baseline, leading to inconsistent predictions.

Evidence (verbatim from paper)

The complete HELM model achieves the overall best or second-best results, with the highest AUPRC on UCM (0.904) and the lowest Ranking Loss across all datasets (0.022, 0.017, 0.006, 0.024).

Citation

@misc{stoimchev2026helm,
  title={HELM: Hierarchical and Explicit Label Modeling with Graph Learning for Multi-Label Image Classification},
  author={Stoimchev et al. (2026)},
  year={2026},
  note={arXiv:2603.11783}
}

• arXiv: 2603.11783