Sim1 Tshirt Fold Eval

sim1-tshirt-fold-eval

SIM1: Physics-Aligned Simulator as Zero-Shot Data Scaler in Deformable Worlds — Zhou et al. (2026) (arXiv:2604.08544, 2026)

What this evaluates

Evaluates a robot policy's ability to perform structured deformable manipulation (t-shirt folding) in real-world settings after being trained exclusively on simulation data. It probes sim-to-real transfer, out-of-domain robustness to environmental shifts, and data scaling efficiency.

Datasets

• SIM1 T-shirt Folding — total ?; splits: test (30); repo https://github.com/InternRobotics/SIM1

Metrics

• success (primary) — range: percent
  • Percentage of trials where the garment reaches the target folded configuration without dropping or unfolding. Calculated as (successful_trials / total_trials) * 100.

Input / output format

Input: RGB camera views of the workspace, robot kinematic state, and garment configuration.

Output: Robot joint commands or end-effector trajectories for dual-arm manipulation.

Scoring recipe

def compute_success(predictions, gold):
    successful = 0
    for trial in predictions:
        if trial.reaches_target_folded_config() and not trial.drops_or_unfolds():
            successful += 1
    return (successful / len(predictions)) * 100

Common pitfalls

• Confusing the 1:15 synthetic-to-real data equivalence ratio with absolute success rates.
• Failing to account for the π_0.5 vs π_0 pretraining difference, which causes π_0 to fail completely in low-data regimes.
• Not distinguishing between in-domain evaluation (identical setup) and out-of-domain generalization (spatial, texture, lighting, viewpoint shifts).

Evidence (verbatim from paper)

For the representative π_0.5 setting, real data reach average success 97%, while policies trained on sim-teleoperated data achieve 87%, a marginal gap of 10%. This suggests that physics-aligned simulation provides supervision of considerable fidelity to match real-world training with controlled data volumes.

Citation

@misc{zhou2026sim1,
  title={SIM1: Physics-Aligned Simulator as Zero-Shot Data Scaler in Deformable Worlds},
  author={Zhou et al. (2026)},
  year={2026},
  note={arXiv:2604.08544}
}

• arXiv: 2604.08544