Hierarchical Visual Recognition Eval

hierarchical-visual-recognition-eval

Taxonomy-Aware Representation Alignment for Hierarchical Visual Recognition with Large Multimodal Models — He et al. (2026) (CVPR 2026, 2026)

What this evaluates

Evaluates a model's ability to perform fine-grained, taxonomy-aware visual recognition by predicting hierarchical biological labels (order, family, genus, species) from images. It specifically probes whether the model maintains logical consistency across taxonomic levels while accurately identifying leaf-level species, including generalization to unseen/novel categories.

Datasets

• iNaturalist-2021 — total ?; splits: train (-1), val (-1)
• TerraIncognita — total ?; splits: known (-1), novel (-1)

Metrics

• Hierarchical Consistent Accuracy (HCA) (primary) — range: [0, 1]
  • HCA = (1/N) * Σ_{i=1}^{N} Π_{j=1}^{L^i} 1[f_θ(x^i; Y_j) = y^i_j]. Computes the proportion of test samples whose predicted taxonomic path exactly matches the ground truth from root to leaf.
• Leaf-Level Accuracy (Acc_leaf) — range: [0, 1]
  • Acc_leaf = (1/N) * Σ_{i=1}^{N} 1[f_θ(x^i; Y_L) = y^i_L]. Measures discriminative ability at the most fine-grained (leaf) level.
• Point-Overlap Ratio (POR) — range: [0, 1]
  • POR = (1/N) * Σ_{i=1}^{N} (Σ_{j=1}^{L_i} 1[f_θ(x_i; Y_j) = y^i_j]) / L_i. Averages the proportion of correctly predicted nodes across the hierarchy, allowing partial correctness.
• Strict Point-Overlap Ratio (S-POR) — range: [0, 1]
  • S-POR = (1/N) * Σ_{i=1}^{N} (1/L_i) * max_{1≤a≤b≤L_i} [(b-a+1) * Π_{j=a}^{b} 1[f_θ(x_i; Y_j) = y^i_j]]. Rewards only the longest contiguous segment of correct predictions normalized by depth.
• Top Overlap Ratio (TOR) — range: [0, 1]
  • TOR = (1/N) * Σ_{i=1}^{N} (1/(L_i-1)) * Σ_{j=1}^{L_i-1} 1[f_θ(x_i; Y_j) = y^i_j] * 1[f_θ(x_i; Y_{j+1}) = y^i_{j+1}]. Evaluates pairwise consistency between consecutive taxonomic layers.
• F1 Score — range: [0, 1]
  • Harmonic mean of precision and recall computed at the Order and Family levels. Used for TerraIncognita where full taxonomic labels are missing for novel species.

Input / output format

Input: Resized image (328×328) paired with a text prompt/question specifying a taxonomic level (order, family, genus, or species). Evaluated in a 1-shot VQA setting.

Output: Predicted taxonomic label(s) for the specified level(s). For TerraIncognita, the model returns a label or 'Unknown' if uncertain.

Scoring recipe

def compute_hca(predictions, golds, hierarchy_depths):
    correct = 0
    for pred, gold, depth in zip(predictions, golds, hierarchy_depths):
        if len(pred) < depth:
            pred = pred + [None] * (depth - len(pred))
        if all(p == g for p, g in zip(pred, gold)):
            correct += 1
    return correct / len(predictions)

Common pitfalls

• HCA requires an exact match across the entire taxonomic path from root to leaf, making it significantly stricter than leaf-level accuracy.
• TerraIncognita novel species lack complete taxonomic annotations, so evaluation is restricted to Order and Family levels only.
• Input images are strictly resized to 328×328 with zero data augmentation, which differs from standard training pipelines and may impact generalization benchmarks.

Evidence (verbatim from paper)

Hierarchical Consistent Accuracy (HCA). This metric is defined as HCA = (1/N) sum_{i=1}^{N} prod_{j=1}^{L^{i}} 1[f_{ heta}(x^{i}; Y_{j}) = y^{i}{j}]. Here, N is the number of test samples, L^{i} is the depth of the hierarchy for the i-th input x^{i}, and Y{j} denotes the label set at level j. HCA computes the proportion of samples whose predicted paths exactly match the ground truth from root to leaf. It is therefore a stricter criterion than flat accuracy and serves as our primary evaluation metric for hierarchical classification.

Citation

@misc{he2026tara,
  title={Taxonomy-Aware Representation Alignment for Hierarchical Visual Recognition with Large Multimodal Models},
  author={He et al. (2026)},
  year={2026},
  note={CVPR 2026}
}

• arXiv: 2603.00431