# Benchmark Author Variants

> Use when a benchmark item or judged run needs a reusable variant pack from the repository's canonical metamorphic and adversarial families, with explicit invariants, drift budget, safe-failure rules, and execution-ready transformed input packages.

- Skill: `loogacystudio/benchmark-author-variants` (Agent Skill, multi-file: 3 files)
- Install (CLI): `npx skillmds@latest add loogacystudio/benchmark-author-variants`
- Raw SKILL.md: https://api.skillmd.com/api/skills/loogacystudio/benchmark-author-variants/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Finance & Business
- Author: LoogacyStudio (https://skillmd.com/u/loogacystudio)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/loogacystudio/benchmark-author-variants

---


# Benchmark Author Variants

Use this skill when a benchmark needs metamorphic or adversarial expansion that can be rerun and judged consistently.

It covers **variant-pack authoring**. It does not execute variants, score them, or decide whether observed drift is acceptable.

For shared benchmark framework rules — capability registry, corpus discovery, corpus adapter contract, lifecycle boundaries, and run-record finalization conventions — defer to `benchmark-core`.

## Purpose

Use it to:

- author metamorphic variants from the shared `M1–M5` families
- author adversarial variants from the shared `A1–A6` families
- keep hard invariants, soft invariants, drift budget, and safe-failure rules explicit
- package transformed inputs so they are ready for execution and later judging
- hand off the executed variant runs to `benchmark-judge-run` instead of judging them inline

## Use this skill when

Typical requests:

- creating the first variant pack for a benchmark item
- expanding a route-sensitive item with invariance checks
- expanding a fragile item with adversarial stress cases
- translating a judged weakness into a bounded variant bundle

### Trigger examples

- "Help me create metamorphic variants for B006"
- "Build an adversarial variant pack"
- "Add hard invariants and a drift budget to this benchmark"
- "Create a variant bundle focused on route drift"

## Do not use this skill when

Do not use this skill when:

- the task is to execute the variant rather than author it
- the request is to judge observed drift rather than create the variant
- the variant idea is actually a benchmark mutation proposal for promotion review
- the request is only for a tiny wording correction in an existing variant file

## Pattern

- Primary pattern: **Generator**
- Secondary pattern: **Tool Wrapper**

Why this fit works:

- the main job is to generate a structured variant bundle
- it also packages the repository's shared family rules, invariance expectations, and stress discipline
- a full Pipeline would add ceremony without enough value because execution and judging remain downstream roles

## Inputs

Collect or infer these inputs when you can:

- base benchmark ID and title
- task family
- base route expectation
- expected artifact type
- why variant work is being requested now
- whether metamorphic, adversarial, or both modes are in scope
- known fragile findings or failure modes from prior judged runs

If some inputs are missing, make the smallest safe assumptions and mark them explicitly.

## Workflow

Use the same sequence each time.

### 1. Frame the variant goal

State what the variant pack should reveal.

Examples:

- route stability under paraphrase
- robustness under misleading evidence
- validation-plan persistence under presentation change
- safe-failure behavior under partial visibility

Do not start from random transformations. Start from the diagnostic purpose.

### 2. Choose the variant mode and family set

Select only the smallest useful family or families:

- metamorphic: `M1` paraphrase, `M2` context-order, `M3` noise tolerance, `M4` terminology substitution, `M5` presentation-style
- adversarial: `A1` injection-in-artifact, `A2` conflicting constraints, `A3` misleading evidence, `A4` partial visibility, `A5` tool degradation, `A6` stale-context contamination

Keep one main stressor per variant unless a compound test is explicitly justified.

### 3. Define the contract for each variant

For every variant, specify:

- variant type and family
- purpose
- main transformation or stressor
- route expectation
- expected invariant or safe-failure rule
- hard invariants
- soft invariants
- allowed drift budget

Do not leave drift interpretation implicit.

### 4. Package the transformed input cleanly

Create an execution-ready transformed input package.

It should be usable by a runner without reconstructing the scenario from scratch.

### 5. Return the variant bundle

Use `assets/variant-bundle-template.md` as the final output skeleton.

## Output contract

Return the result using `assets/variant-bundle-template.md` in the same section order.

Output rules:

- choose only justified variant families
- keep hard invariants and soft invariants explicit
- keep drift budget explicit for metamorphic variants
- keep safe-failure expectation explicit for adversarial variants
- keep transformed input packages execution-ready
- avoid random prompt churn dressed up as evaluation rigor

## Companion files

- `assets/variant-bundle-template.md` — final variant-bundle output structure
- `references/variant-sources.md` — canonical family and workflow sources
- `../benchmark-core/references/benchmark-skill-lifecycle.md` — shared handoff map for benchmark skills

## Validation

A good result should satisfy all of the following:

- each variant has a clear diagnostic purpose
- family choice is explicit and justified
- invariants and safe-failure rules are explicit
- drift budget is stated where metamorphic behavior matters
- transformed inputs are runnable without reconstruction theater
- one main stressor is preserved per variant unless compound stress is explicitly intended

## Common pitfalls

- generating random paraphrases with no diagnostic value
- omitting hard invariants and then calling the result an invariance check
- mixing multiple stressors into one muddy variant
- leaving route expectation implicit on route-sensitive items
- writing a judge note instead of an execution-ready variant pack

## Completion rule

This skill is complete when the agent has:

- defined the diagnostic goal of the variant work
- selected the right family or families
- authored explicit variant contracts with invariants or safe-failure rules
- packaged execution-ready transformed inputs
- returned a reusable variant bundle for downstream execution and judging