Metafold Garment Folding Eval

metafold-garment-folding-eval

MetaFold: Language-Guided Multi-Category Garment Folding Framework via Trajectory Generation and Foundation Model — Chen et al. (2025) (arXiv:2503.08372, 2025)

What this evaluates

This benchmark evaluates a robotic framework's ability to fold various garments according to language instructions. It probes the model's spatial-temporal trajectory generation, action prediction accuracy, and generalization across different garment categories and unseen language prompts.

Datasets

• MetaFold dataset — total ?; splits: test (-1)
• CLOTH3D — total ?; splits: zero-shot (-1)

Metrics

• Rectangularity — range: [0, 1]
  • The ratio of the final garment area to its bounding rectangle. It assesses how well the folded garment resembles a rectangle.
• Area Ratio — range: [0, 1]
  • The ratio of the final garment area to the initial garment area. Lower values indicate a more compact folded configuration.
• Success Rate (primary) — range: [0, 1]
  • The fraction of instances where rectangularity exceeds a specified threshold and area ratio falls below a certain threshold. Averaged over all test samples.

Input / output format

Input: Point cloud representation of the garment mesh and natural language folding instructions.

Output: Sequence of robot actions or a 3D trajectory for manipulating the garment to achieve the folded state.

Scoring recipe

def compute_metrics(predictions, gold):
    final_area = get_mesh_area(predictions)
    init_area = get_mesh_area(gold)
    rect_area = get_bounding_rect_area(predictions)
    rectangularity = final_area / rect_area
    area_ratio = final_area / init_area
    # Thresholds are task-specific but not explicitly stated in the text
    success = (rectangularity > RECT_THRESH) and (area_ratio < AREA_THRESH)
    return rectangularity, area_ratio, success

Common pitfalls

• Success Rate thresholds are not explicitly defined in the paper, making exact replication difficult.
• Area Ratio is a 'lower-is-better' metric, which can be counterintuitive compared to standard accuracy metrics.
• All evaluations are conducted in Isaac Sim; results may not directly translate to real-world fabric dynamics.

Evidence (verbatim from paper)

We employed multiple metrics to evaluate the folding quality from various perspectives. Rectangularity. The ratio of the final garment area to its bounding rectangle. We use this metric to assess how well the folded garment resembles a rectangle, providing a measure of the folding quality. Area Ratio. The ratio of the final garment area to the initial garment area. This metric indicates the folding quality, as each fold that brings points closer to garment boundaries reduces the area, leading to a more compact configuration. Success Rate. The success rate is measured by the rectangularity exceeding a specified threshold and the area ratio falling below a certain threshold.

Citation

@misc{chen2025metafold,
  title={MetaFold: Language-Guided Multi-Category Garment Folding Framework via Trajectory Generation and Foundation Model},
  author={Chen et al. (2025)},
  year={2025},
  note={arXiv:2503.08372}
}

• arXiv: 2503.08372