Pairwise Interaction Eval

pairwise-interaction-eval

Detection and Recognition: A Pairwise Interaction Framework for Mobile Service Robots — Liang et al. (2026) (arXiv:2602.22346, 2026)

What this evaluates

Evaluates a robot's ability to detect interacting human pairs and classify their coarse-grained interaction types (e.g., walking, standing, sitting together) using bounding box geometry and optical flow, without relying on costly skeleton-based pose estimation.

Datasets

• JRDB — total ?; splits: train (-1), test (-1)
• Collective Activity Dataset (CAD) — total ?; splits: train (-1), test (-1)
• Lawnmower Dataset — total ?; splits: test (-1)

Metrics

• Accuracy (primary) — range: [0, 1] or percent
  • Overall fraction of correctly classified pairwise interactions: sum of correct predictions divided by total number of predictions.
• Mean Per-Class Accuracy (MPCA) — range: [0, 1] or percent
  • Average of per-class recall across C classes: (1/C) * sum(TP_c / N_c). Accounts for class imbalance.
• Macro F1-score — range: [0, 1]
  • Average of per-class F1 scores: (1/C) * sum(2 * P_c * R_c / (P_c + R_c)), where P_c and R_c are per-class precision and recall.

Input / output format

Input: Video frames with bounding box annotations for individuals, plus optical flow and geometric cues (spatial proximity).

Output: Predicted pairwise interaction type (e.g., walking, standing, sitting together) and group membership/activity labels.

Scoring recipe

def compute_metrics(preds, golds, C):
    N = len(golds)
    accuracy = sum(1 for p, g in zip(preds, golds) if p == g) / N
    
    tp = [0] * C
    n = [0] * C
    for p, g in zip(preds, golds):
        n[g] += 1
        if p == g:
            tp[g] += 1
            
    mpca = sum(tp[c] / n[c] for c in range(C) if n[c] > 0) / C
    
    macro_f1 = 0
    for c in range(C):
        p_c = tp[c] / (tp[c] + (n[c] - tp[c])) if (tp[c] + (n[c] - tp[c])) > 0 else 0
        r_c = tp[c] / n[c] if n[c] > 0 else 0
        f1_c = 2 * p_c * r_c / (p_c + r_c) if (p_c + r_c) > 0 else 0
        macro_f1 += f1_c
    macro_f1 /= C
    return accuracy, mpca, macro_f1

Common pitfalls

• Class imbalance is significant, requiring weighted sampling and focal loss during training, which affects metric interpretation.
• Group-level activity is derived via majority voting over pairwise predictions, which can mask individual interaction errors and inflate group accuracy.
• Zero-shot evaluation on the lawnmower dataset suffers from domain shift (platform motion, low resolution, grass occlusion), making direct comparison with static-camera benchmarks misleading.

Evidence (verbatim from paper)

We report Accuracy, Mean Per-Class Accuracy (MPCA), and Macro F1-score to account for class imbalance:

| | $\text{Macro-F1}=\frac{1}{C}\sum_{c=1}^{C}\frac{2\cdot P_{c}\cdot R_{c}}{P_{c}+R_{c}}$ | | (14) |

where $P_{c}=TP_{c}/(TP_{c}+FP_{c})$ and $R_{c}=TP_{c}/(TP_{c}+FN_{c})$ are per-class precision and recall, respectively.

Citation

@misc{liang2026detection,
  title={Detection and Recognition: A Pairwise Interaction Framework for Mobile Service Robots},
  author={Liang et al. (2026)},
  year={2026},
  note={arXiv:2602.22346}
}

• arXiv: 2602.22346