Sega Layout Eval

sega-layout-eval

SEGA: A Stepwise Evolution Paradigm for Content-Aware Layout Generation with Design Prior — Wang et al. (2025) (arXiv:2510.15749, 2025)

What this evaluates

Evaluates a model's ability to generate content-aware graphic layouts from background images and instructions. It probes spatial reasoning, adherence to design principles (alignment, overlap, occlusion), and aesthetic quality.

Datasets

• PKU — total 9974; splits: (unstated)
• CGL — total 60548; splits: (unstated)
• Crello — total 23182; splits: (unstated)

Metrics

• Ali (primary) — range: other (↑/↓ directionality)
  • Alignment metric calculated based on design principle (iv): elements should align with each other. Higher is better.
• Ove — range: other (↑/↓ directionality)
  • Overlap metric calculated based on design principle (i): elements are not overlaid. Lower is better.
• Und_l — range: other (↑/↓ directionality)
  • Underlay large metric based on principle (ii): non-underlay elements should be covered by the underlay. Higher is better.
• Und_s — range: other (↑/↓ directionality)
  • Underlay small metric based on principle (ii): non-underlay elements should be covered by the underlay. Higher is better.
• Read — range: other (↑/↓ directionality)
  • Readability metric based on text element placement and coverage. Lower is better.
• Occ — range: other (↑/↓ directionality)
  • Occlusion metric based on principle (iii): elements should not cover the object in the background. Lower is better.
• S_Mean — range: other (↑/↓ directionality)
  • Mean aesthetic score computed from pre-trained models (DL, QL, TV, IO). Higher is better.
• Inference Time — range: seconds
  • Wall-clock seconds required to generate the layout, including coarse estimation and iterative refinement steps.

Input / output format

Input: Background image I, task instruction D, and conditional constraint C.

Output: Layout L = {e_1, ..., e_k} where each element e = (c, l, t, w, h), plus an evaluation text E^Pred describing design principle violations.

Scoring recipe

def compute_metrics(pred_layout, gt_layout):
    # Rule-based metrics from 4 design principles
    overlap = calculate_bbox_intersection(pred_layout)
    alignment = check_coordinate_proximity(pred_layout, threshold=5.0)
    underlay = check_z_order_coverage(pred_layout)
    occlusion = check_background_covering(pred_layout)
    readability = estimate_text_element_readability(pred_layout)
    
    # Aesthetic scores from external models
    aesthetic = compute_aesthetic_scores(pred_layout)
    
    return {
        'Ali': alignment, 'Ove': overlap, 'Und_l': underlay,
        'Und_s': underlay, 'Read': readability, 'Occ': occlusion,
        'S_Mean': aesthetic.mean(), 'Inference Time': wall_clock_time()
    }

Common pitfalls

• Rule-based metrics (Ali, Ove, etc.) rely on implicit coordinate thresholds for alignment and overlap detection that are not explicitly defined in the main text.
• Inference time includes both coarse estimation and iterative refinement steps, which can vary depending on the number of iterations chosen.
• Aesthetic scores (S_DL, S_QL, etc.) are computed by external pre-trained models and may not correlate perfectly with human preference.

Evidence (verbatim from paper)

Concretely, we briefly summarize the principles for layout generation as four points: (i) Elements are not overlaid. (ii) Non-underlay elements should be covered by the underlay. (iii) Elements should not cover the object in the background. (iv) Elements should align with each other. The corresponding evaluation is defined based on the rule-based metric calculation. Experimental results report metrics including Ali↓, Ove↓, Und_l↑, Und_s↑, Read↓, Occ↓, and aesthetic scores S_Mean↑.

Citation

@misc{wang2025sega,
  title={SEGA: A Stepwise Evolution Paradigm for Content-Aware Layout Generation with Design Prior},
  author={Wang et al. (2025)},
  year={2025},
  note={arXiv:2510.15749}
}

• arXiv: 2510.15749