qhjqhj00 human-flow-eval
Learning Human Optical Flow — Ranjan et al. (2018) (arXiv:1806.05666, 2018)
What this evaluates
Evaluates the accuracy and inference speed of optical flow estimation networks specifically for human motion. It probes a model's ability to capture fine-grained, small-scale displacements typical of human limbs and body parts against complex or layered backgrounds.
Datasets
- Human Flow — total ?; splits: train (135153), test (10867)
Metrics
AEPE(primary) — range: pixels- Average End Point Error: the mean L2 Euclidean distance between predicted and ground truth optical flow vectors across all pixels in the test set.
Evaluation Time— range: seconds- Wall-clock time in seconds required to process a single pair of frames on an NVIDIA Titan X GPU.
Input / output format
Input: Pair of consecutive RGB frames (images) containing human subjects in motion.
Output: 2D optical flow field (H x W x 2 tensor) representing pixel-wise displacement vectors between the two frames.
Scoring recipe
def compute_aepe(pred_flow, gt_flow):
# pred_flow and gt_flow are HxWx2 tensors
diff = pred_flow - gt_flow
l2_norms = torch.sqrt(torch.sum(diff**2, dim=-1) + 1e-6)
return torch.mean(l2_norms).item()
Common pitfalls
- Models trained on general datasets (e.g., FlowNetS) often fail on the small motions typical of human limbs, yielding results close to a zero-flow baseline.
- Layered optical flow methods (e.g., PCA-layers) heavily depend on accurate foreground-background segmentation, which is frequently imperfect in complex scenes.
- Traditional non-learning methods (e.g., LDOF, EpicFlow) tend to produce blurred flow fields that miss sharp boundaries at hands and legs, and are significantly slower than real-time inference.
Evidence (verbatim from paper)
We compare the average End Point Errors (EPEs) of competing methods in Table 1(b) along with the time for evaluation. Human motion is complex and general optical flow methods fail to capture it. Our trained network outperforms previous methods, and SPyNet [5] in particular, in terms of average EPE on the Human Flow Dataset.
Citation
@misc{ranjan2018learninghumanopticalflow,
title={Learning Human Optical Flow},
author={Ranjan et al. (2018)},
year={2018},
note={arXiv:1806.05666}
}
- arXiv: 1806.05666