qhjqhj00 knowgen-eval
Gen-Searcher: Reinforcing Agentic Search for Image Generation — Feng et al. (2026) (arXiv:2603.28767, 2026)
What this evaluates
Evaluates an agentic search agent's ability to gather external knowledge and reference images to enhance text-to-image generation for knowledge-intensive, real-world prompts. It measures how well the agent's search-grounded prompts improve visual correctness, text accuracy, faithfulness, and aesthetics compared to direct generation.
Datasets
- KnowGen — total ?; splits: test (-1)
Metrics
K-Score(primary) — range: [0, 100]- Average of four dimension scores: Visual correctness, Text accuracy, Faithfulness, and Aesthetics. Scores are computed per subset (Science & Knowledge, Pop Culture & News) and then averaged for the overall K-Score. Reported on a 0-100 scale.
Input / output format
Input: Original text prompt describing the desired image.
Output: Search-grounded prompt and selected reference images, which are passed to a downstream image generator to produce the final image. If search fails, the original prompt is used as a fallback.
Scoring recipe
def compute_k_score(predictions, gold):
# Evaluate generated image against original prompt
visual_correctness = score_visual_attributes(predictions, gold)
text_accuracy = score_text_rendering(predictions, gold)
faithfulness = score_content_alignment(predictions, gold)
aesthetics = score_composition_quality(predictions)
# Average the four dimensions
subset_score = (visual_correctness + text_accuracy + faithfulness + aesthetics) / 4
# Average across subsets (Science & Knowledge, Pop Culture & News)
overall_k_score = average(subset_scores)
return overall_k_score
Common pitfalls
- If the agent fails to produce a final search-grounded prompt due to context limits or tool failures, the system falls back to the original prompt, which can artificially lower scores if not tracked.
- Aesthetics scores may decrease when integrating multiple retrieved reference images, as the generator struggles with composition despite improved factual grounding.
- Downstream generator limitations (e.g., multi-subject consistency, poor text rendering) can cause generation failures even when the search agent correctly retrieves evidence.
Evidence (verbatim from paper)
The overall K-Score is averaged over the Science & Knowledge and Pop Culture & News subsets.
Citation
@misc{feng2026gensearcher,
title={Gen-Searcher: Reinforcing Agentic Search for Image Generation},
author={Feng et al. (2026)},
year={2026},
note={arXiv:2603.28767}
}
- arXiv: 2603.28767