# Sarena Icon Eval

> Evaluates a model's ability to generate scalable vector graphics (SVG) from text prompts and reference images, measuring visual fidelity, semantic alignment, structural success, and code efficiency. Use when the user wants to benchmark on SArena-Icon, or asks about evaluating this task. Reports SR.

- Skill: `qhjqhj00/sarena-icon-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/sarena-icon-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/sarena-icon-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/sarena-icon-eval

---


# sarena-icon-eval

> Reliable Reasoning in SVG-LLMs via Multi-Task Multi-Reward Reinforcement Learning — Haomin Wang et al. (2026) (arXiv:2603.16189, 2026)

## What this evaluates

Evaluates a model's ability to generate scalable vector graphics (SVG) from text prompts and reference images, measuring visual fidelity, semantic alignment, structural success, and code efficiency.

## Datasets

- **SArena-Icon** — total ?; splits: test (-1)

## Metrics

- `SR` **(primary)** — range: percent
  - Success Rate: percentage of generated SVGs that successfully render without syntax or structural errors.
- `CLIP-T2I` — range: other
  - CLIP Text-to-Image similarity score between the text prompt and the rendered SVG image.
- `CLIP-I2I` — range: other
  - CLIP Image-to-Image similarity score between the reference image and the rendered SVG image.
- `FID` — range: other
  - Fréchet Inception Distance measuring the distribution difference between real and generated SVG images.
- `DINO` — range: [0, 1]
  - DINO similarity score between the reference/generated image and the ground truth.
- `SSIM` — range: [0, 1]
  - Structural Similarity Index measuring perceptual similarity between images.
- `LPIPS` — range: [0, 1]
  - Learned Perceptual Image Patch Similarity measuring perceptual distance between images.
- `Tokens` — range: other
  - Number of tokens in the generated SVG code, measuring code efficiency.

## Input / output format

**Input**: Text prompt (for Text-to-SVG) or reference image (for Image-to-SVG).

**Output**: SVG code string.

## Scoring recipe

```python
def compute_metrics(predictions, gold):
    rendered_imgs = [render_svg(svg) for svg in predictions]
    sr = sum(1 for img in rendered_imgs if img.is_valid()) / len(predictions)
    clip_t2i = clip_similarity(text_prompts, rendered_imgs)
    clip_i2i = clip_similarity(ref_images, rendered_imgs)
    fid = frechet_distance(real_imgs, rendered_imgs)
    dino = dino_similarity(ref_imgs, rendered_imgs)
    ssim = structural_similarity(ref_imgs, rendered_imgs)
    lpips = perceptual_distance(ref_imgs, rendered_imgs)
    tokens = [count_tokens(svg) for svg in predictions]
    return {'SR': sr, 'CLIP-T2I': clip_t2i, 'CLIP-I2I': clip_i2i, 'FID': fid, 'DINO': dino, 'SSIM': ssim, 'LPIPS': lpips, 'Tokens': tokens}
```

## Common pitfalls

- Metrics are computed on the rasterized/rendered output of the SVG, not the raw code.
- Success Rate (SR) requires a parser or renderer to verify SVG validity before computing visual metrics.

## Evidence (verbatim from paper)

> CTRL-S achieves leading performance on both the Text-to-SVG and Image-to-SVG tasks on the SArena-Icon benchmark. For Text-to-SVG, CTRL-S attains the highest CLIP-T2I score of 25.944... For Image-to-SVG, our model obtains the best results across the DINO, SSIM, and LPIPS metrics compared to mainstream general VLMs and SVG-LLMs.

## Citation

```bibtex
@misc{wang2026ctrls,
  title={Reliable Reasoning in SVG-LLMs via Multi-Task Multi-Reward Reinforcement Learning},
  author={Haomin Wang et al. (2026)},
  year={2026},
  note={arXiv:2603.16189}
}
```

- arXiv: 2603.16189