Long Term Motion Eval

long-term-motion-eval

It's a Matter of Time: Three Lessons on Long-Term Motion for Perception — Davison et al. (2026) (arXiv:2602.14705, 2026)

What this evaluates

Evaluates long-term motion representations derived from dense point-tracking against image-based baselines across five perceptual tasks. It probes temporal generalization, motion representation efficiency, and the ability to capture spatio-temporal dynamics for classification and regression.

Datasets

• SSV2 (Temporal Dataset subset) — total ?; splits: unspecified (-1)
• Jester — total ?; splits: unspecified (-1)
• VB100 — total ?; splits: unspecified (-1)
• RAVDESS — total ?; splits: unspecified (-1)
• MITFabric — total ?; splits: unspecified (-1)
• ADVIO — total ?; splits: unspecified (-1)

Metrics

• classification accuracy (primary) — range: [0, 1]
  • Proportion of correctly predicted class labels out of the total number of instances.
• Pearson correlation coefficient — range: [-1, 1]
  • Measures the linear correlation between predicted and ground-truth continuous values for material properties (stiffness and area weight).
• RMSE translational RPE — range: other
  • Root-mean-squared error of the relative translational pose error, averaged over all time-step predictions.
• RMSE rotational RPE — range: other
  • Root-mean-squared error of the relative rotational pose error, averaged over all time-step predictions.

Input / output format

Input: Preprocessed video clips (32, 90, 120, or 300 frames depending on the dataset) containing either pixel frames or dense point-track coordinates.

Output: Class labels for action, object, and emotion tasks; continuous values for material properties (stiffness, area weight); and 6-DoF pose parameters (translation and orientation) for spatial understanding.

Scoring recipe

if task in ['action', 'object', 'emotion']:
    return sum(pred == gold) / len(gold)
elif task == 'material':
    return pearsonr(pred, gold)  # per property
elif task == 'spatial':
    trans_err = sqrt(mean((pred_trans - gold_trans)**2))
    rot_err = sqrt(mean((pred_rot - gold_rot)**2))
    return trans_err, rot_err

Common pitfalls

• Using the full SSV2 dataset instead of the specified 'Temporal Dataset' subset, which includes classes where temporal information is not essential.
• Ignoring dataset-specific preprocessing requirements (e.g., exact frame counts, fps normalization, clip lengths) which drastically alter motion representation quality.
• Evaluating material properties and spatial pose as classification tasks instead of regression, or using accuracy instead of Pearson correlation/RMSE.

Evidence (verbatim from paper)

Averaging over all time-step predictions, we then report the root-mean-squared-error (RMSE) translational RPE and RMSE rotational RPE as our final evaluation metrics.

Citation

@misc{davison2026longtermmotion,
  title={It's a Matter of Time: Three Lessons on Long-Term Motion for Perception},
  author={Davison et al. (2026)},
  year={2026},
  note={arXiv:2602.14705}
}

• arXiv: 2602.14705