Body Tracking

Body Tracking Skill

Use this skill to debug body tracking and face-body association issues.

When to Use

• Screen time gaps when cast is on screen but face not visible
• Face-body association errors
• Re-ID matching issues after occlusion
• Debugging person detection coverage
• Track fusion problems

Sub-agents

Key Skills

Run body tracking pipeline

Run the sandbox pipeline on an episode (writes artifacts under data/manifests/{ep_id}/body_tracking/).

from pathlib import Path

from FEATURES.body_tracking.src.body_tracking_runner import BodyTrackingRunner

runner = BodyTrackingRunner(
    episode_id="my-episode",
    config_path=Path("config/pipeline/body_detection.yaml"),
    fusion_config_path=Path("config/pipeline/track_fusion.yaml"),
)
runner.run_full_pipeline()

Fuse face ↔ body tracks

from FEATURES.body_tracking.src.track_fusion import TrackFusion

fusion = TrackFusion()
identities = fusion.fuse_tracks(face_tracks=face_tracks, body_tracks=body_tracks)

Compute body-only screentime deltas

python -m FEATURES.body_tracking --episode-id <EP_ID> --stage compare

Config Reference

File: config/pipeline/body_detection.yaml

File: config/pipeline/track_fusion.yaml

Common Issues

Missing body detections

Cause: Person not detected by YOLO

Check: Detection confidence in logs

Fix: Lower confidence threshold:

person_detection:
  confidence_threshold: 0.40  # default is 0.50

Wrong face-body association

Cause: IoU too loose or multiple people close together

Check: association_confidence in track metadata

Fix: Increase IoU threshold:

track_fusion:
  association_iou_thresh: 0.60  # default is 0.50

Re-ID fails after occlusion

Cause: Clothing change or long gap

Check: reid_similarity between pre/post occlusion

Fix: Lower Re-ID threshold or increase gap tolerance:

track_fusion:
  reid_similarity_thresh: 0.60  # default is 0.70
  handoff:
    max_gap_seconds: 45  # default is 30

Body track ID switches

Cause: Tracker losing track through occlusion

Check: body_id_switch_rate in metrics

Fix: Increase track buffer:

person_tracking:
  track_buffer: 150  # default is 120 (5 seconds)

Diagnostic Output

{
  "identity_id": "ID_001",
  "face_track_ids": [42, 256],
  "body_track_ids": [100127, 100345],
  "screen_time": {
    "face_visible_duration": 125.5,
    "body_only_duration": 45.2,
    "total_duration": 170.7
  },
  "associations": [
    {
      "face_track": 42,
      "body_track": 100127,
      "method": "iou",
      "confidence": 0.85
    },
    {
      "face_track": null,
      "body_track": 100127,
      "method": "reid_handoff",
      "confidence": 0.72
    }
  ]
}

Key Files

Related Skills

• face-alignment - Face alignment debugging
• pipeline-insights - General pipeline debugging