Curriculum Dpo++ Eval

curriculum-dpo++-eval

Curriculum-DPO++: Direct Preference Optimization via Data and Model Curricula for Text-to-Image Generation — Florinel-Alin Croitoru et al. (arXiv:2602.13055, 2026)

What this evaluates

Evaluates text-to-image generation models on their ability to align generated images with text prompts, produce visually appealing outputs, and match human preferences. It tests the effectiveness of curriculum-based fine-tuning strategies on standard generative benchmarks.

Datasets

• D1 (Black-ICLR-2024) — total 74250; splits: train (67500), test (6750)
• D2 (DrawBench) — total 110000; splits: train (100000), test (10000)
• D3 (Pick-a-Pic) — total 150000; splits: test (500)

Metrics

• Text Alignment (primary) — range: [-1, 1]
  • Cosine similarity between Sentence-BERT embeddings of the original prompt and a caption generated by LLaVA from the image.
• Aesthetics — range: other
  • Score from the LAION Aesthetics Predictor, a linear model based on CLIP representations.
• Human Preference — range: other
  • Score from HPSv2, a CLIP model fine-tuned on human-ranked image pairs.

Input / output format

Input: Text prompts (SVO templates, DrawBench prompts, or Pick-a-Pic test prompts)

Output: Generated images (256x256 for SD, 768x768 for LCM)

Scoring recipe

def compute_metrics(prompts, images):
    scores = {'text_alignment': [], 'aesthetics': [], 'human_preference': []}
    for prompt, image in zip(prompts, images):
        caption = llava.generate_caption(image)
        scores['text_alignment'].append(cosine_similarity(sbert(prompt), sbert(caption)))
        scores['aesthetics'].append(laion_aesthetics.predict(image))
        scores['human_preference'].append(hpsv2.predict(image))
    return {k: np.mean(v) for k, v in scores.items()}

Common pitfalls

• Using different reward models or generation settings across methods, which the paper explicitly warns against to ensure fair comparison.
• Confusing the reward-model-free training strategy (which uses prompt perturbation/masking to infer pair difficulty) with the evaluation metrics themselves.

Evidence (verbatim from paper)

To measure text alignment, we compute the cosine similarity between Sentence-BERT embeddings of the original prompt and a caption generated by LLaVA. For visual appeal, we utilize the LAION Aesthetics Predictor, a linear model based on CLIP representations. Finally, human preference is estimated using HPSv2, a CLIP model fine-tuned on human-ranked image pairs.

Citation

@misc{croitoru2026curriculumdpo++,
  title={Curriculum-DPO++: Direct Preference Optimization via Data and Model Curricula for Text-to-Image Generation},
  author={Florinel-Alin Croitoru et al.},
  year={2026},
  note={arXiv:2602.13055}
}

• arXiv: 2602.13055