Deco 50 Eval

deco-50-eval

DECO: Decoupled Multimodal Diffusion Transformer for Bimanual Dexterous Manipulation with a Plugin Tactile Adapter — Li et al. (2026) (arXiv:2602.05513, 2026)

What this evaluates

Evaluates a robot policy's ability to perform bimanual dexterous manipulation tasks under varying levels of tactile dependency. It probes visual-propriocceptive coordination, dynamic object interaction, and contact-rich force control.

Datasets

• DECO-50 — total ?; splits: train (-1), test (-1)

Metrics

• Success Rate (primary) — range: percent
  • Calculated as the number of successful task completions divided by the total number of attempts. Two aggregate metrics are reported: Success Rate 1 (average across all four tasks) and Success Rate 2 (average across the two contact-rich tasks: Waste Disposal and Assembly). For Waste Disposal, success requires completing all three sequential stages.

Input / output format

Input: Synchronized multimodal observations: active binocular camera images, tactile sensor signals, and robot proprioceptive state.

Output: Continuous bimanual action commands (e.g., joint positions or end-effector poses) for the dual-arm robot.

Scoring recipe

def compute_success_rate(successes, attempts):
    return successes / attempts

def compute_aggregate_sr(task_success_rates):
    return sum(task_success_rates) / len(task_success_rates)

# Success Rate 1: average over all 4 tasks
# Success Rate 2: average over Waste Disposal and Assembly

Common pitfalls

• Assuming tactile sensing is universally required; the paper demonstrates it only significantly boosts performance on contact-rich tasks (Waste Disposal, Assembly).
• Misinterpreting aggregate metrics: Success Rate 1 averages all tasks, while Success Rate 2 specifically averages only the contact-rich tasks.
• Overlooking stage-wise dependencies in Waste Disposal, where overall success requires completing all three sequential stages (open lid, pick/throw, close lid).

Evidence (verbatim from paper)

Success Rate 1: Average success rate on all tasks. Success Rate 2: Average success rate on contact-rich tasks, which are Waste Disposal and Assembly.

Citation

@misc{li2026deco,
  title={DECO: Decoupled Multimodal Diffusion Transformer for Bimanual Dexterous Manipulation with a Plugin Tactile Adapter},
  author={Li et al. (2026)},
  year={2026},
  note={arXiv:2602.05513}
}

• arXiv: 2602.05513