Layerd Eval

layerd-eval

LayerD: Decomposing Raster Graphic Designs into Layers — Suzuki et al. (2025) (arXiv:2509.25134, 2025)

What this evaluates

Evaluates a model's ability to decompose raster graphic designs into a sequence of re-editable layers. It measures visual reconstruction quality and the number of edits required to match a ground-truth layer structure, accounting for the ill-posed nature of layer ordering.

Datasets

• Crello — total ?; splits: train (19478), val (1852), test (1971)

Metrics

• RGB L1 (primary) — range: [0, 255]
  • Mean absolute error between predicted and ground-truth pixel RGB values across the image.
• Alpha IoU (primary) — range: [0, 1]
  • Intersection over Union of predicted and ground-truth alpha (transparency) masks.
• Edit Distance (DTW) — range: integer
  • Number of edits (insertions/deletions) to align predicted layer sequence with ground truth, computed using Dynamic Time Warping to handle ambiguous layer ordering.

Input / output format

Input: Raster graphic design image (aspect ratio preserved, shorter side resized to 512 pixels).

Output: Ordered sequence of layers, each containing a segmentation mask, alpha channel, color map, and z-index (depth order).

Scoring recipe

def compute_metrics(pred_layers, gt_layers):
    # Align sequences using DTW to handle ordering ambiguity
    aligned_pred, aligned_gt = dtw_align(pred_layers, gt_layers)
    rgb_l1 = 0
    alpha_iou = 0
    for p, g in zip(aligned_pred, aligned_gt):
        rgb_l1 += np.mean(np.abs(p.rgb - g.rgb))
        alpha_iou += jaccard_index(p.alpha, g.alpha)
    return rgb_l1 / len(aligned_pred), alpha_iou / len(aligned_pred)

Common pitfalls

• Transparent layers are explicitly excluded from evaluation.
• Text layers are evaluated separately from non-text layers.
• Edit distance uses Dynamic Time Warping (DTW) to handle the ill-posed, ambiguous nature of ground-truth layer structures.
• Metrics are reported as a function of the maximum number of allowed edits (0 to 5).

Evidence (verbatim from paper)

We show visual quality metrics (RGB L1, Alpha IoU) as the maximum number of allowed edits increases.

Citation

@misc{suzuki2025layerd,
  title={LayerD: Decomposing Raster Graphic Designs into Layers},
  author={Suzuki et al. (2025)},
  year={2025},
  note={arXiv:2509.25134}
}

• arXiv: 2509.25134