# Layerd Eval

> Evaluates a model's ability to decompose raster graphic designs into a sequence of re-editable layers. It measures visual reconstruction quality and the number of edits required to match a ground-truth layer structure, accounting for the ill-posed nature of layer ordering. Use when the user wants to benchmark on Crello, or asks about evaluating this task. Reports RGB L1, Alpha IoU.

- Skill: `qhjqhj00/layerd-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/layerd-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/layerd-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/layerd-eval

---


# layerd-eval

> LayerD: Decomposing Raster Graphic Designs into Layers — Suzuki et al. (2025) (arXiv:2509.25134, 2025)

## What this evaluates

Evaluates a model's ability to decompose raster graphic designs into a sequence of re-editable layers. It measures visual reconstruction quality and the number of edits required to match a ground-truth layer structure, accounting for the ill-posed nature of layer ordering.

## Datasets

- **Crello** — total ?; splits: train (19478), val (1852), test (1971)

## Metrics

- `RGB L1` **(primary)** — range: [0, 255]
  - Mean absolute error between predicted and ground-truth pixel RGB values across the image.
- `Alpha IoU` **(primary)** — range: [0, 1]
  - Intersection over Union of predicted and ground-truth alpha (transparency) masks.
- `Edit Distance (DTW)` — range: integer
  - Number of edits (insertions/deletions) to align predicted layer sequence with ground truth, computed using Dynamic Time Warping to handle ambiguous layer ordering.

## Input / output format

**Input**: Raster graphic design image (aspect ratio preserved, shorter side resized to 512 pixels).

**Output**: Ordered sequence of layers, each containing a segmentation mask, alpha channel, color map, and z-index (depth order).

## Scoring recipe

```python
def compute_metrics(pred_layers, gt_layers):
    # Align sequences using DTW to handle ordering ambiguity
    aligned_pred, aligned_gt = dtw_align(pred_layers, gt_layers)
    rgb_l1 = 0
    alpha_iou = 0
    for p, g in zip(aligned_pred, aligned_gt):
        rgb_l1 += np.mean(np.abs(p.rgb - g.rgb))
        alpha_iou += jaccard_index(p.alpha, g.alpha)
    return rgb_l1 / len(aligned_pred), alpha_iou / len(aligned_pred)
```

## Common pitfalls

- Transparent layers are explicitly excluded from evaluation.
- Text layers are evaluated separately from non-text layers.
- Edit distance uses Dynamic Time Warping (DTW) to handle the ill-posed, ambiguous nature of ground-truth layer structures.
- Metrics are reported as a function of the maximum number of allowed edits (0 to 5).

## Evidence (verbatim from paper)

> We show visual quality metrics (RGB L1, Alpha IoU) as the maximum number of allowed edits increases.

## Citation

```bibtex
@misc{suzuki2025layerd,
  title={LayerD: Decomposing Raster Graphic Designs into Layers},
  author={Suzuki et al. (2025)},
  year={2025},
  note={arXiv:2509.25134}
}
```

- arXiv: 2509.25134