# Benchmark Enhancement Capture

> Capture a proven AI-assisted recovery into the responsible design, local tool fork, plugin program, or expert skill without adding a policy layer.

- Skill: `vibeic/benchmark-enhancement-capture` (Agent Skill, multi-file: 3 files)
- Install (CLI): `npx get-skill add vibeic/benchmark-enhancement-capture`
- Raw SKILL.md: https://api.skillmd.com/api/skills/vibeic/benchmark-enhancement-capture/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: vibeic (https://skillmd.com/u/vibeic)
- Updated: 2026-11-15
- Page: https://skillmd.com/skills/vibeic/benchmark-enhancement-capture

---


# Capture a proven recovery

Read `CHARTER.md`. Capture a recovery when AI review finds and resolves a real
problem in a native result. The purpose is to make the responsible tool,
producer, or expert craft better on the next applicable input.

## Captured rule: only the input determines what may be learned

Owner ruling, 2026-11-13: SKILL SHOULD BE GENERAL AND NO-CHEATING.
For a benchmark recovery, only what the complete supplied INPUT states can
justify a behavioral requirement. A testbench, scorer outcome or golden answer
cannot establish intent, even after scoring or when its expected answer has
been anonymized. A PASS is measurement evidence, never a source of requirements.

Retain general engineering techniques with their input conditions. Put no
benchmark or problem name, remembered answer or checker-derived default in
author-facing knowledge. Where INPUT is silent or ambiguous, the capture is
that the design goes to AI authoring with the exact ambiguity named. AI uses
the complete input, records choices left free by it, and does not present a
choice as a source requirement. A hidden checker cannot settle that choice.
Measured chip cases, including their native tool, numbers and design identity,
remain engineering evidence. This prohibition concerns benchmark problem
answers in author-facing guidance, not hardware reference measurements.
Review a proposed capture against newer file history and owner rulings before
restoring an older rule; superseded instructions stay superseded.

## Establish the recovery

1. Preserve the visible input, native tool command, original limit, observed failure, and actual corrected result outside the plugin repository.
2. Explain the cause in terms of the failed requirement and its actual consumer.
3. Distinguish a design error, a local tool defect, and a plugin defect. Fix the responsible one.
4. Verify the fix through the native tool and relevant existing tests without moving a limit.
5. State where the fix generalizes and what remains unmeasured.

For benchmarks, use `benchmark_dispatch.py` for Program First, independent
blind AI review and backup, and official scoring. Authors and reviewers read
only the provided prompt and context. They do not read hidden oracle,
harness, or golden contents. A rejection requires a prompt-derived executable
challenge, and a repair must satisfy that challenge and the normal product
verification before a fresh independent review.

## Put the fix with its owner

Consult `benchmark/CAPTURE_ROUTING.json` for the responsible program and skill.
For native flow glue, use the actual caller of the tool in `flow_engine.py`,
`flow_native_adapters.py`, or the relevant native helper. For native tool
behavior, fix the local tool fork. For document extraction or RTL generation,
fix its existing producer. Do not add a checker merely to repeat another
checker or to satisfy an obsolete wrapper.

A reusable technical lesson belongs in the applicable expert skill or IC
Expert knowledge, with the same input-only boundary as the original work.
Keep the lesson general. Do not copy a design's identity, customer data,
source text, hidden expected answer, or source-specific values into general
control logic. Preserve the IC Expert DB and per-layer craft.

Capture the reproducible discrepancy, affected source fact or native result, owning design/tool/plugin fix and the verification performed. Keep this evidence outside the plugin repository when required by the task.

## Verify and deliver

Run the next owner-requested verification on the changed product path. Keep
any owner-requested cohort size or repetition count. Preserve independent
blind actors and the official scorer, and disclose substitutions or missing
measurements. Do not claim a speedup, a benchmark score, or whole-design
acceptance from metadata alone.

Keep run results separate from product code. Use the current task's
authorization for changes, external reports, messages, PRs, pushes, and landing.
The owner's known landing workflow is `~/claude-leader/land_one.sh`; retain
`gen_program_inventory.py`, `nda_diff_scan_check.py`, and
`tools/release/assign_version.py` and use their actual instructions. This skill
does not add another release policy or authorize a landing on its own.

## Smallest fix of the original problem (Charter C10, owner 2026-11-14)

Owner: "fixing PR/issue 不要越搞越複雜，用最小的方式、最小的改動去修正原來的問題." Before a fix, state the
original problem in one sentence; keep main's existing path and change only what that problem needs. Review asks
only: is the original problem fixed, is anything worse than main, and is the fix general and no-cheating? A defect found in behaviour main already has
is a separate known gap, written down, never folded into the same PR. After about two SEND BACKs, or when a PR keeps
adding new mechanism (parsers, analyses, inventories, readers) to answer counterexamples, stop and restart from main
as the smallest fix of the original problem.
A smallest fix enters the plugin or a skill only if it is also general and no-cheating (owner 2026-11-14):
it holds for any design meeting the same condition, names no benchmark, problem, design or testbench, and takes
nothing from a testbench, reference answer, scorer or score. A design-specific fix belongs in that design, not in
the product.

