Zero Shot Human Classification Eval

zero-shot-human-classification-eval

Human Pose Descriptions and Subject-Focused Attention for Improved Zero-Shot Transfer in Human-Centric Classification Tasks — Muhammad Saif Ullah Khan et al. (2024) (arXiv:2403.06904, 2024)

What this evaluates

Evaluates the zero-shot transfer capability of a vision-language model on human-centric classification tasks, including activity recognition, age grouping, and emotion recognition, using pose-grounded text descriptions and subject-focused attention.

Datasets

• Stanford40 — total ?; splits: test (-1)
• Emotic — total ?; splits: test (-1)
• LAGENDA-Body — total ?; splits: test (-1)
• LAGENDA-Face — total ?; splits: test (-1)
• UTKFace — total ?; splits: test (-1)
• FER+ — total ?; splits: test (-1)

Metrics

• top-k accuracy (primary) — range: [0, 1]
  • Measures whether the ground-truth class label appears within the top k predicted classes by the model. k is set to 1 for age and emotion tasks, and 3 for activity recognition.

Input / output format

Input: RGB image paired with a natural language pose description (text prompt).

Output: Predicted class label (activity, age group, or emotion) ranked by confidence score.

Scoring recipe

def top_k_accuracy(predictions, labels, k):
    correct = 0
    for pred, label in zip(predictions, labels):
        top_k_preds = pred.argsort()[-k:][::-1]
        if label in top_k_preds:
            correct += 1
    return correct / len(labels)

Common pitfalls

• Age labels are originally numerical but must be explicitly binned into categorical groups (adult, teenager, kid) before evaluation.
• The evaluation is strictly zero-shot: models are trained only on the source dataset (MPII) and tested on unseen target datasets without fine-tuning.
• Different k values are used per task category (k=1 for age/emotion, k=3 for activity), so results are not directly comparable without normalization.

Evidence (verbatim from paper)

Tab. 2 presents our quantitative results. We report the top-k accuracy for image-based activity classification, age classification, and emotion recognition.

Citation

@misc{khan2024humanposedescriptions,
  title={Human Pose Descriptions and Subject-Focused Attention for Improved Zero-Shot Transfer in Human-Centric Classification Tasks},
  author={Muhammad Saif Ullah Khan et al. (2024)},
  year={2024},
  note={arXiv:2403.06904}
}

• arXiv: 2403.06904