Skill Release Pipeline
Ship a reproducible package and an honest result, not a folder plus a marketing claim.
Release contract
- Name one real job and the observable behavior the skill should improve.
- Check the target repository for an incumbent with the same trigger. Evaluate against that incumbent when one exists, not an empty baseline.
- Build the smallest package that can change the target behavior. Keep deterministic operations in
scripts/ and conditional detail in references/.
- Run static gates before spending model calls: strict front matter, package tests, secret scan, license and provenance checks, and exact file hashes.
- Create representative tasks from real work. Separate authoring cases from held-out cases. Make the verifier fail a known-bad output and pass both short and thorough known-good outputs.
- Use the repository's
skill-eval-loop for blind paired runs. Require at least three valid pairs for an adoption decision. Use more tasks when the public claim depends on a confidence interval.
- Confirm that the treatment actually loaded the candidate. A run where it did not load is invocation evidence, not quality evidence.
- Publish the package identity, task definitions, outputs, verifier results, grader verdicts, failures, and limitations. Raw private inputs remain local unless the user explicitly approves disclosure.
- Publish the observed status even when it is negative or inconclusive. Never convert a smoke test, launch review, or attractive example into an efficacy claim.
- Merge only after CI passes, an independent review finds no blocking issue, and the public record identifies the exact shipped package hash.
Result states
supported: the predefined statistical gate passes and no safety or package gate fails.inconclusive: the run completed but the predefined evidence bar was not reached.negative: the baseline performed materially better or the candidate introduced a blocking regression.not_run: no controlled comparison exists.invalid: parity, loading, blinding, or verifier integrity failed.
After every package or evidence change, run scripts/refresh_derivation.py <skill-directory>, then run scripts/release_check.py <skill-directory>. The first command binds the release record to the exact bytes on disk. The second fails closed on inventory drift, hash drift, malformed metadata, invalid result states, and common secret patterns. Neither replaces behavioral evaluation or human review.
Publishing to GitHub, GetEdge, a package registry, or social media is an external action. Use authorization from the current request only for the destinations it actually covers.
1---2name: skill-release-pipeline3description: Take an Agent Skill from a concrete user problem through package creation, behavioral evaluation, evidence binding, and publication. Use when asked to build and publish a new skill, turn a skill idea into an evaluated release, or run the full Edge skill release workflow.4---56# Skill Release Pipeline78Ship a reproducible package and an honest result, not a folder plus a marketing claim.910## Release contract11121. Name one real job and the observable behavior the skill should improve.132. Check the target repository for an incumbent with the same trigger. Evaluate against that incumbent when one exists, not an empty baseline.143. Build the smallest package that can change the target behavior. Keep deterministic operations in `scripts/` and conditional detail in `references/`.154. Run static gates before spending model calls: strict front matter, package tests, secret scan, license and provenance checks, and exact file hashes.165. Create representative tasks from real work. Separate authoring cases from held-out cases. Make the verifier fail a known-bad output and pass both short and thorough known-good outputs.176. Use the repository's `skill-eval-loop` for blind paired runs. Require at least three valid pairs for an adoption decision. Use more tasks when the public claim depends on a confidence interval.187. Confirm that the treatment actually loaded the candidate. A run where it did not load is invocation evidence, not quality evidence.198. Publish the package identity, task definitions, outputs, verifier results, grader verdicts, failures, and limitations. Raw private inputs remain local unless the user explicitly approves disclosure.209. Publish the observed status even when it is negative or inconclusive. Never convert a smoke test, launch review, or attractive example into an efficacy claim.2110. Merge only after CI passes, an independent review finds no blocking issue, and the public record identifies the exact shipped package hash.2223## Result states2425- `supported`: the predefined statistical gate passes and no safety or package gate fails.26- `inconclusive`: the run completed but the predefined evidence bar was not reached.27- `negative`: the baseline performed materially better or the candidate introduced a blocking regression.28- `not_run`: no controlled comparison exists.29- `invalid`: parity, loading, blinding, or verifier integrity failed.3031After every package or evidence change, run `scripts/refresh_derivation.py <skill-directory>`, then run `scripts/release_check.py <skill-directory>`. The first command binds the release record to the exact bytes on disk. The second fails closed on inventory drift, hash drift, malformed metadata, invalid result states, and common secret patterns. Neither replaces behavioral evaluation or human review.3233Publishing to GitHub, GetEdge, a package registry, or social media is an external action. Use authorization from the current request only for the destinations it actually covers.
getedgehq