Switch Justdance Eval

switch-justdance-eval

Switch-JustDance: Benchmarking Whole Body Motion Tracking Policies Using a Commercial Console Game — Jeonghwan Kim et al. (arXiv:2511.17925, 2025)

What this evaluates

Evaluates whole-body motion tracking policies for humanoid robots by measuring how well they synchronize with reference choreography from the commercial game Just Dance. It probes tracking accuracy, stability over long-horizon motions, and movement smoothness compared to human baselines.

Datasets

• Just Dance Routines — total 5; splits: test (5)

Metrics

• Just Dance Score (JDS) (primary) — range: [0, 13333]
  • In-game score computed from Joy-Con motion tracking, reflecting synchronization with reference choreography. Maximum score is 13,333 per song.
• Mean Per Joint Position Error (MPJPE, mm) — range: other
  • Average joint position error between the robot's state estimation and the retargeted reference trajectory in the robot's torso frame. Reported as Active (pre-fall) and All (entire sequence). Lower values indicate better tracking.
• Success Rate (SR, %) — range: percent
  • Percentage of trials where the controller completes the entire dance without falling or overheating. Higher values indicate better stability.
• Smoothness — range: other
  • Average joint jerk (rad/s^3) and acceleration (rad/s^2) computed using finite differences on joint angles. Lower values indicate smoother, more human-like motion.

Input / output format

Input: Retargeted reference motion trajectories (either streaming smoothed or offline dynamic) and robot internal state estimation.

Output: Robot joint commands/trajectories to track the reference motion.

Scoring recipe

def compute_metrics(robot_joints, ref_joints, falls_at_frame=None, overheated=False):
    jds = get_in_game_score(robot_joints)  # Max 13333
    mpjpe_all = mean(np.linalg.norm(robot_joints - ref_joints, axis=1))
    mpjpe_active = mean(np.linalg.norm(robot_joints[:falls_at_frame] - ref_joints[:falls_at_frame], axis=1)) if falls_at_frame else mpjpe_all
    sr = 1.0 if not falls_at_frame and not overheated else 0.0
    jerk = mean(np.abs(np.diff(robot_joints, n=3)))
    acc = mean(np.abs(np.diff(robot_joints, n=2)))
    return {'JDS': jds, 'MPJPE_Active': mpjpe_active, 'MPJPE_All': mpjpe_all, 'SR': sr, 'Jerk': jerk, 'Acc': acc}

Common pitfalls

• MPJPE must be reported separately for Active (pre-fall) and All frames depending on whether the robot falls mid-dance.
• Just Dance Score (JDS) is only applicable in real-world hardware setups; it cannot be computed in simulation due to the lack of a physical Joy-Con.
• Streaming smoothed (Smo) input introduces phase lag but suppresses high-frequency jitter, while offline dynamic (Dyn) preserves timing but challenges controller agility.

Evidence (verbatim from paper)

We use four complementary metrics to evaluate general humanoid motion-tracking controllers, each capturing a distinct aspect of performance:

• Just Dance Score (JDS): The in-game score from the Nintendo Switch serves as our primary performance indicator. Computed from Joy-Con motion tracking, it reflects how well the robot synchronizes with the reference choreography. The max score for every song in Just Dance is 13,333.

Citation

@misc{kim2025switchjustdance,
  title={Switch-JustDance: Benchmarking Whole Body Motion Tracking Policies Using a Commercial Console Game},
  author={Jeonghwan Kim et al.},
  year={2025},
  note={arXiv:2511.17925}
}

• arXiv: 2511.17925