Detailmaster Eval

detailmaster-eval

Long-Text-to-Image Generation via Compositional Prompt Decomposition — Huang et al. (2026) (arXiv:2604.18258, 2026)

What this evaluates

Evaluates a text-to-image model's ability to faithfully render long, descriptive prompts. It probes fine-grained semantic alignment across character presence, attributes, spatial relationships, and scene composition, as well as overall aesthetic and alignment quality using preference models.

Datasets

• DetailMaster — total ?; splits: test (-1)

Metrics

• CharacterPresence (primary) — range: percent
  • Percentage of prompts where the generated image successfully contains the specified characters.
• CharacterAttributes — range: percent
  • Accuracy of matching character features to the prompt, computed separately for object, animal, and person categories.
• CharacterLocation — range: percent
  • Accuracy of correctly positioning specified characters in the generated image.
• SceneAttributes — range: percent
  • Accuracy of adhering to background, lighting, and style instructions in the prompt.
• SpatialRelation — range: percent
  • Accuracy of reflecting specified spatial relationships between characters.
• CLIPScore — range: other
  • Reference-free metric measuring text-image alignment using CLIP embeddings.
• DenScore — range: other
  • Preference score evaluating overall text-image alignment.
• PickScore — range: other
  • Preference score based on user preference data for text-to-image generation.
• VQAScore — range: other
  • Score from a multimodal LLM evaluating finer detail alignment between image and prompt.
• HPSv3 — range: other
  • Human Preference Score v3, evaluating wide-spectrum human preference for generated images.

Input / output format

Input: Long-form text prompts (paragraphs, averaging ~285 tokens) describing scenes, characters, attributes, and spatial relationships.

Output: A single RGB image generated from the text prompt.

Scoring recipe

def compute_accuracy(predictions, gold):
    correct = 0
    for pred_img, gold_attrs in zip(predictions, gold):
        if check_attributes_match(pred_img, gold_attrs):
            correct += 1
    return (correct / len(predictions)) * 100

def compute_preference_score(predictions, gold_prompts):
    scores = [get_model_score(pred_img, prompt) for pred_img, prompt in zip(predictions, gold_prompts)]
    return sum(scores) / len(scores)

Common pitfalls

• Models trained on short prompts degrade sharply on prompts exceeding 500 tokens, so length-stratified evaluation is necessary.
• Simple sentence splitting for decomposition loses global context and yields broken scenes compared to learned compositional methods.
• Evaluations must be reported separately for different backbone architectures (e.g., SD-1.5 vs. Qwen-Image) as performance varies significantly.

Evidence (verbatim from paper)

Table 1 summarizes the benchmark evaluations of DetailMaster, where we examine PRISM against specialized Long-Text-to-Image generation methods and SOTA baselines. PRISM-SD1.5 outperforms other methods by 2.33% on CharacterPresence and 1.53% on CharacterLocation, demonstrating the PRISM’s efficiency in processing descriptive paragraphs.

Citation

@misc{huang2026prism,
  title={Long-Text-to-Image Generation via Compositional Prompt Decomposition},
  author={Huang et al. (2026)},
  year={2026},
  note={arXiv:2604.18258}
}

• arXiv: 2604.18258