Harness Engineering

Harness Engineering

Overview

Use this skill to translate harness engineering principles into concrete repository changes that improve agent speed, correctness, and maintainability.

Workflow

1) Assess the current harness

• Inspect AGENTS.md, contributor docs, CI workflows, and repo scripts.
• Map the actual path from prompt to merged change.
• List concrete failure modes: ambiguous instructions, repeated mistakes, slow reviews, merge conflicts, stale docs, or noisy CI.

2) Define operating boundaries

• Separate decisions by layer:
  • Human layer: architecture, taste, policy, and final risk acceptance.
  • Agent layer: implementation, refactors, test updates, and routine maintenance.
• Define non-negotiable constraints before coding starts:
  • Required tests and linters.
  • Directory ownership and file-scoping rules.
  • Branch, PR, and commit conventions.

3) Make the repository the system of record

• Encode process in files, not tribal knowledge.
• Prefer deterministic entry points:
  • Single source for setup.
  • Canonical command wrappers (make, just, or scripts).
  • Stable templates for PRs, issues, and specs.
• Remove conflicting guidance and duplicate docs.

4) Increase agent legibility

• Prefer one clear pattern over many equivalent ones.
• Standardize naming, folder layouts, and module boundaries.
• Add concise examples for fragile tasks.
• Eliminate hidden behavior and implicit coupling.

5) Add mechanical guardrails

• Enforce checks automatically:
  • Fast local checks for iteration.
  • CI gates for merge readiness.
  • Static analysis for known failure classes.
• Fail with actionable messages that point to exact fixes.
• Keep checks strict enough to prevent regressions, but fast enough for daily use.
• For any verifier or generated QA artifact, check verifier integrity before trusting a pass result.

6) Optimize merge throughput

• Encourage smaller, composable changes instead of large batches.
• Isolate independent work streams to reduce collisions.
• Define explicit integration rules for overlapping files.
• Prefer rapid merge and follow-up fixes over long-lived divergence.

7) Build entropy-control loops

• Schedule regular cleanup work: dead code removal, doc pruning, flaky test fixes.
• Track recurring failure patterns and encode permanent safeguards.
• Retire obsolete scripts, prompts, and templates quickly.

Deliverable format

• Harness snapshot: current bottlenecks with evidence.
• Proposed harness changes: files to add or edit and why.
• Guardrail plan: checks to enforce and where they run.
• Rollout plan: phased adoption with fallback strategy.
• Entropy plan: recurring cleanup cadence and ownership.

Verifier integrity model

Use this model whenever changing tests, manifests, verifier profiles, generated artifacts, QA summaries, or completion gates.

• Oracle truth: the verifier defines correctness explicitly and cannot pass because the assertion is vacuous, normalized too broadly, or missing the negative case.
• Coverage truth: the verifier reaches the intended files, rows, paths, states, profiles, and generated outputs.
• Artifact truth: summaries, reports, and persisted artifacts cannot claim pass when required completeness or source evidence failed.
• Failure-signal truth: the exact broken condition produces a non-zero or blocking result with an actionable message.
• Binding truth: each verifier names the requirement, policy, or contract it proves, and each required contract names the verifier that owns it.

For generated or tool-driven pipelines, also check input normalization, output validation, retry/idempotency, redaction, source-to-runtime inventory alignment, and fail-closed behavior when storage or semantic inventories drift.

Classify failure semantics before implementation:

• deterministic failure: invalid state or broken contract; fail immediately.
• retryable failure: transient dependency or transport issue; retry only within a bounded policy.
• silent discard: forbidden unless a spec explicitly permits it and defines observability.

References

• For detailed principles, anti-patterns, and direct implementation checklists, read references/harness-engineering-principles.md.