Hico Hoi Eval

hico-hoi-eval

HAKE: Human Activity Knowledge Engine — Li et al. (2019) (arXiv:1904.06539, 2019)

What this evaluates

Evaluates human-object interaction recognition by decomposing activities into atomic body part states and reasoning hierarchically. Probes the model's ability to handle long-tail data and few-shot learning scenarios through compositional part-state representations.

Datasets

• HICO — total 47774; splits: train (38116), test (9658)

Metrics

• mAP (primary) — range: percent
  • Mean Average Precision across all interaction categories. Computed as the average of the Area Under the Precision-Recall curve for each category, then averaged across all categories.

Input / output format

Input: RGB images containing humans and objects, with annotations for body parts, part verbs, and object parts forming activity triplets ⟨body_part, part_verb, object_part⟩.

Output: Ranked list of predicted activity triplets with confidence scores.

Scoring recipe

def compute_map(predictions, ground_truth, num_classes):
    aps = []
    for c in range(num_classes):
        preds_c = sorted([p for p in predictions if p['class'] == c], key=lambda x: x['score'], reverse=True)
        gt_c = set(g['id'] for g in ground_truth if g['class'] == c)
        tp, fp = 0, 0
        precisions, recalls = [], []
        for pred in preds_c:
            if pred['id'] in gt_c: tp += 1
            else: fp += 1
            precisions.append(tp / (tp + fp))
            recalls.append(tp / len(gt_c))
        aps.append(trapezoidal_ap(precisions, recalls))
    return sum(aps) / num_classes

Common pitfalls

• mAP gains are reported relative to a specific baseline (Pairwise), not absolute state-of-the-art across all methods.
• Few-shot results are averaged over cumulative training size thresholds (<1, <5, <10 images) rather than standard fixed k-shot splits.
• HAKE-GT results use ground-truth part states, representing an upper bound rather than a fully end-to-end pipeline.

Evidence (verbatim from paper)

HICO contains 38,116 images in train set and 9,658 images in test set. ... From Tab.[2] we can find that our method achieve 7.2 mAP gain over the state-of-the-art result on HICO.

Citation

@misc{li2019hake,
  title={HAKE: Human Activity Knowledge Engine},
  author={Li et al. (2019)},
  year={2019},
  note={arXiv:1904.06539}
}

• arXiv: 1904.06539