Lunara Aesthetic Ii Eval

lunara-aesthetic-ii-eval

Moonworks Lunara Aesthetic II: An Image Variation Dataset — Wang et al. (2026) (arXiv:2602.01666, 2026)

What this evaluates

Evaluates the quality, contextual variation isolation, aesthetic appeal, and identity preservation of the Lunara Aesthetic II image variation dataset compared to other web-scale datasets.

Datasets

• Lunara-II-Variations — total 2854; splits: full (2854)
• Lunara-I — total 2000; splits: full (2000)
• CC3M — total 1000; splits: full (1000)
• LAION-2B-Aesthetic — total 1000; splits: full (1000)
• WIT — total 1000; splits: full (1000)

Metrics

• Axis specificity — range: [0, 1]
  • Measures how well each contextual axis is isolated from others, computed using conditional co-occurrence probabilities. Higher values indicate greater isolation.
• Prompt alignment (Cohen's d) — range: other
  • Effect size for token-level similarity between base and modified prompts. More negative values correspond to larger prompt edits.
• Axis entropy (normalized) — range: [0, 1]
  • Captures the diversity of prompt realizations within each axis. Lower values indicate stronger axis isolation, higher values reflect broader expressive variation.
• LAION Aesthetics v2 score (primary) — range: [0, 10]
  • CLIP-based predictor score approximating aggregate human judgments of visual appeal. Scores typically range from 0 to 10.
• Identity Stability — range: [1, 5]
  • Human-rated 5-point Likert scale assessing whether the core identity of the subject remains consistent across anchor and variation images.
• Target Attribute Realization — range: percent
  • Percentage of variation sets where the intended contextual change is clearly expressed and correctly realized according to the target label.

Input / output format

Input: Anchor images paired with their contextual variants, along with associated text prompts and target axis labels.

Output: Metric scores including axis specificity, Cohen's d, normalized entropy, LAION aesthetic scores, human Likert ratings, and target attribute realization percentages.

Scoring recipe

def compute_metrics(images, prompts, labels, human_ratings):
    # Automated
    specificity = conditional_cooccurrence_prob(labels) / prior_prob(labels)
    prompt_d = cohens_d(tokenize(prompts['base']), tokenize(prompts['varied']))
    entropy = normalize_entropy(count_prompt_realizations(labels))
    aesthetic = clip_aesthetic_v2.predict(images)
    # Human
    identity_stability = mean([r for r in human_ratings['identity']])
    realization = sum(1 for s in sets if human_yes_no(s) == s.target) / len(sets) * 100
    return specificity, prompt_d, entropy, aesthetic, identity_stability, realization

Common pitfalls

• High aesthetic quality can bias human raters, inflating perceived identity stability scores on the Likert scale.
• Semantic deltas (e.g., color tone vs. mood) are often intertwined, causing ambiguity in axis isolation and assessment.
• Prompt phrasing can inadvertently alter composition or viewpoint, confounding targeted contextual evaluations.

Evidence (verbatim from paper)

We assess controlled contextual variation using prompt-level and distributional analyses. Table [2] reports automated metrics evaluating the isolation, expressiveness, and prompt-level impact of each contextual variation axis. ... Identity Stability. Researchers assess whether the core identity of the subject (e.g., the same object instance or entity) remains consistent across the anchor image and all variations. This criterion is necessary for attributing differences in model behavior to the intended contextual change rather than unintended identity drift. For each variation pair (an original image and a variation), evaluate using 5-point Likert scale (1: poor stability, the variation does not retain original identity, 5: the variation image respects original identity).

Citation

@misc{wang2026lunara,
  title={Moonworks Lunara Aesthetic II: An Image Variation Dataset},
  author={Wang et al. (2026)},
  year={2026},
  note={arXiv:2602.01666}
}

• arXiv: 2602.01666