# Composition Evaluate

> Compare FlowGraph composition policies with fixed tasks, requested-versus-realized resources, failure injection, and an evidence claim gate

- Skill: `jmagly/composition-evaluate` (Agent Skill)
- Install (CLI): `npx skillmds@latest add jmagly/composition-evaluate`
- Raw SKILL.md: https://api.skillmd.com/api/skills/jmagly/composition-evaluate/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: jmagly (https://skillmd.com/u/jmagly)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/jmagly/composition-evaluate

---


# Composition evaluation

Use this skill when a composition policy needs reproducible comparison against
a single-pass baseline.

1. Start from `benchmarks/composition-policy-benchmark.v1.json` and preserve
   fixed tasks, settings, budgets, seeds, metrics, and thresholds.
2. Run `aiwg composition benchmark <manifest.json>`; use `--raw-out` and
   `--summary-out` to retain both evidence layers.
3. Compare success-conditioned cost and latency, not unconditioned cheap
   failures. Review speed-of-accuracy and strict-LCM-versus-adaptive deltas.
4. Require an independent or human evaluation path and inspect self-judge bias.
5. Review every failure-injection outcome and recovery receipt.
6. Keep synthetic conformance distinct from provider evidence. Do not open the
   claim gate without repeated trusted runs, independent evaluation, confidence
   intervals, and task-family replication.

Composition graphs remain `flow.aiwg.io/v1alpha1` `FlowGraph`; do not introduce
a fourth-level DNS API group. Do not request or persist private chain-of-thought.

@implements #2118