# Worldlens Eval

> worldlens-eval

- Skill: `qhjqhj00/worldlens-eval` (Agent Skill)
- Install (CLI): `npx skillmds@latest add qhjqhj00/worldlens-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/worldlens-eval/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-21
- Page: https://skillmd.com/skills/qhjqhj00/worldlens-eval

---


# worldlens-eval

> WorldLens: Full-Spectrum Evaluations of Driving World Models in Real World — Liang et al. (2025) (arXiv:2512.10958, 2025)

## What this evaluates

Evaluates driving world models across five dimensions: generation quality, 3D/4D reconstruction coherence, action-following capability in closed-loop simulation, downstream perception task utility, and alignment with human preference. It probes geometric consistency, physical plausibility, and functional reliability of synthesized driving scenes.

## Datasets

- **WorldLens** — total ?; splits: test (-1); repo https://github.com/worldbench/WorldLens

## Metrics

- `FID (Subject Fidelity)` — range: [0, inf)
  - Fréchet Inception Distance measuring the distance between feature distributions of real and generated driving frames. Lower is better.
- `Perceptual Discrepancy` — range: [0, 1] or dB
  - Likely LPIPS or PSNR-based metric quantifying visual similarity between generated and ground-truth frames. Lower is better.
- `Geometric Discrepancy / Photometric Error` — range: [0, inf)
  - Errors computed after reconstructing generated videos into 4D Gaussian Fields. Measures floaters, structural instability, and pixel-level reconstruction loss.
- `Route Completion (%)` **(primary)** — range: [0, 100] percent
  - Percentage of simulation episodes where a closed-loop planner successfully navigates the route without collisions or off-road drift.
- `Human Preference Score` — range: [1, 10]
  - Subjective ratings on a 1–10 scale across World Realism, Physical Plausibility, 3D & 4D Consistency, and Behavioral Safety.

## Input / output format

**Input**: Synthesized video sequences or frame-by-frame predictions from driving world models, typically conditioned on ground-truth frames, ego-vehicle states, or action commands.

**Output**: Quantitative metric scores per evaluation aspect, plus human preference ratings on a 1–10 scale and textual rationales for the WorldLens-Agent auto-evaluator.

## Scoring recipe

```python
def compute_metrics(predictions, gold):
    scores = {}
    scores['FID'] = frechet_inception_distance(predictions, gold)
    scores['Perceptual'] = perceptual_similarity(predictions, gold)
    scores['Geo_Error'] = reconstruct_4d_gaussians(predictions).compute_error(gold)
    scores['Route_Completion'] = closed_loop_planner(predictions).success_rate()
    scores['Human_Pref'] = human_annotators.score(predictions, scale=10)
    return scores
```

## Common pitfalls

- Relying solely on perceptual metrics (e.g., FID/LPIPS) without verifying geometric or physical consistency, as texture-rich models often violate physics.
- Evaluating only in open-loop settings; closed-loop simulation exposes severe planner failures (collisions, off-road drift) that open-loop metrics miss.
- Assuming high perceptual quality guarantees downstream task usability; multi-domain training can actually hinder task-specific adaptation.

## Evidence (verbatim from paper)

> As summarized in Table [1], all existing models remain notably below the ‘Empirical Max’, indicating substantial room for improving the visual and temporal realism of driving world models. Although DiST-4D achieves the lowest Perceptual Discrepancy, it underperforms OpenDWM in Subject Fidelity and View Consistency, demonstrating that perceptual metrics alone are insufficient for assessing physically coherent scene generation.

## Citation

```bibtex
@misc{liang2025worldlens,
  title={WorldLens: Full-Spectrum Evaluations of Driving World Models in Real World},
  author={Liang et al. (2025)},
  year={2025},
  note={arXiv:2512.10958}
}
```

- arXiv: 2512.10958