qhjqhj00 cloth-unfolding-grasp-eval
A Dataset and Benchmark for Robotic Cloth Unfolding Grasp Selection: The ICRA 2024 Cloth Competition — Victor-Louis De Gusseme et al. (2025) (ICRA 2024 / arXiv:2508.16749, 2025)
What this evaluates
Evaluates a robot's ability to select effective grasp poses on hanging, folded cloth to maximize unfolding coverage. It probes material-aware perception, geometric reasoning, and policy-based grasp generation under complex folding configurations.
Datasets
- ICRA 2024 Cloth Competition dataset — total 679; splits: train (503), test (176); repo https://github.com/Victorlouisdg/cloth-competition
Metrics
relative coverage(primary) — range: [0, 1]- Ratio of the unfolded cloth's surface area to the maximum achievable surface area (measured from a reference fully unfolded observation). Success rate is also tracked as a binary metric based on coverage thresholds or manual annotation.
Input / output format
Input: Stereo RGB images, depth and confidence maps, colored 3D point cloud, robot joint angles, gripper/base poses, camera intrinsics/extrinsics, and video. Each instance provides a start observation of the hanging cloth and the executed grasp pose.
Output: Grasp pose (position and orientation) for the robot gripper, typically specified as a 4x4 transformation matrix or coordinate frame relative to the world/cloth.
Scoring recipe
def score_grasp(result_obs, reference_obs):
cloth_area = compute_surface_area(result_obs.point_cloud)
max_area = compute_surface_area(reference_obs.point_cloud)
coverage = cloth_area / max_area if max_area > 0 else 0.0
success = coverage > 0.5 # threshold inferred from competition context
return {'coverage': coverage, 'success': success}
Common pitfalls
- Training videos are sometimes incomplete due to recording issues, so evaluation should rely on the 176 fully recorded competition trials.
- The dataset intentionally includes failed grasps and low-coverage attempts; users must filter or account for these rather than assuming all demonstrations are successful.
- Depth-based segmentation is required to remove robot arms and background for accurate surface area calculation, which may vary across different robotic setups.
Evidence (verbatim from paper)
The dataset includes a reference observation for each cloth item. This observation shows the cloth fully unfolded and held in the air, providing a baseline measurement of the maximum achievable surface area for use in evaluation metrics that measure relative coverage.
Citation
@misc{degusseme2025clothcompetition,
title={A Dataset and Benchmark for Robotic Cloth Unfolding Grasp Selection: The ICRA 2024 Cloth Competition},
author={Victor-Louis De Gusseme et al. (2025)},
year={2025},
note={ICRA 2024 / arXiv:2508.16749}
}
- arXiv: 2508.16749