Harness Engineering Copilot

Harness Engineering for GitHub Copilot

Strategies and patterns for building high-leverage harnesses through Copilot's customization system. A harness is the scaffolding — context layering, constraint enforcement, entropy management, and feedback loops — that makes agents reliably productive.

Related skills:

• harness-engineering — generic methodology (tool-agnostic).
• agent-customization — Copilot file formats, syntax, YAML frontmatter, tool aliases, and configuration rules. Defer all "how do I write this file" questions there.

Sources: OpenAI — Harness Engineering | Martin Fowler — Harness Engineering

Use this skill in two modes:

1. Design mode — decide how to layer context, split roles, and add enforcement.
2. Audit mode — review an existing Copilot harness with fixed passes, explicit budgets, and consistency checks.

Strategy 1: Context Layering

Context is a scarce resource. The harness must deliver the right context at the right time — not dump everything into every interaction.

The Three-Tier Context Model

Cross-Check with agent-customization skill

Use the agent-customization decision flow to decide which primitive owns which layer instead of choosing files ad hoc.

Before adding a new file, ask the agent-customization questions in this order:

1. Scope: Is this workspace-shared behavior or a personal preference?
2. Primitive: Is this always-on guidance, on-demand knowledge, a single focused prompt, or a delegated subagent workflow?
3. Discovery: Will Copilot find it from the description, or are you relying on file names and hope?
4. Load cost: Does this belong in auto-loaded context, or should it stay dormant until invoked?

If you cannot answer those four questions clearly, the layer boundary is still underspecified.

Context Layering Best Practices

1. Map, don't manual. Always-on instructions should be a table of contents pointing to deeper sources elsewhere in the repo. If your copilot-instructions.md exceeds ~100 lines, you're overloading Tier 1.
2. One instruction file per architectural boundary. Use applyTo glob patterns that mirror your module/layer structure. When an agent edits src/server/**, it should receive server-layer rules — not frontend rules.
3. Skills as progressive disclosure. Move detailed reference material (DB schemas, API specs, deployment procedures) into skills with references/ folders. The SKILL.md body stays lean; the agent loads references/schema.md only when actually needed.
4. Honor the primitive boundary. If the content is multi-step workflow logic, it probably belongs in a prompt or custom agent, not in always-on instructions. If it is durable domain knowledge, it probably belongs in a skill, not a prompt.
5. Redundancy kills. Never duplicate content across tiers. If a convention is in copilot-instructions.md, don't repeat it in a skill. Use cross-references instead.
6. Pointer chains, not deep nesting. Every reference should be at most one hop from an entry point. If an agent needs to follow AGENTS.md → docs/index.md → docs/design-docs/index.md → docs/design-docs/auth.md, that's too deep. Flatten to AGENTS.md → docs/design-docs/auth.md.
7. Treat description as part of the harness. A skill or agent that cannot be discovered by its description is effectively missing from the harness, even if the file exists.

Context Freshness Strategy

Stale context is worse than no context — agents confidently follow outdated rules.

• CI validation: Add a job that checks cross-references between instruction files and actual repo paths. If an instruction references docs/RELIABILITY.md and the file doesn't exist, fail the build.
• Git-blame-based staleness: Flag instruction files not updated in >90 days alongside the code they govern. If backend.instructions.md hasn't changed but src/server/ has had 50 commits, something is probably stale.
• Changelogs as triggers: When a significant architectural change merges, add "update harness" to the PR checklist. Treat instructions as code that must be kept in sync.

Strategy 2: Multi-Agent Constraint Enforcement

A single omniscient agent is an anti-pattern. Design an agent fleet (agent squad/agent swarm) where each agent has a narrow responsibility and minimal tool access.

The Observer / Actor / Maintainer Triad

Designing an Agent Fleet

Step 1: Identify enforcement surfaces. List every invariant you want to enforce (dependency direction, naming, logging, test coverage, doc freshness). Each surface maps to an Observer agent.

Step 2: Define tool boundaries. For each agent, ask: What's the minimum tool set needed? An architecture reviewer needs read + search, never edit. A test generator needs execute to run tests. Over-provisioning tools undermines the harness.

Step 3: Write agent prompts as enforcement contracts. The agent's markdown body is its enforcement contract — a precise specification of what to check, what to flag, and how to remediate. Structure every enforcement agent prompt as:

1. SCOPE: What files/modules to examine
2. RULES: Numbered invariants to check (reference docs/ for details)
3. PROCESS: Step-by-step verification procedure
4. OUTPUT: Exact format for findings (file, line, violation, remediation)
5. BOUNDARIES: What the agent must NOT do

Step 4: Chain agents for complex workflows. Use subagent invocation (agent tool alias) to compose:

• Planner (read-only) → produces a plan
• Implementor (full tools) → executes the plan
• Reviewer (read-only) → validates the result

This mirrors the "Ralph Wiggum Loop" pattern: agents review each other's work in a feedback loop until all reviewers are satisfied.

Least-Privilege Patterns

Runtime-Specific Tool Assignment

Use runtime to describe the Copilot execution environment of the agent file: VS Code, GitHub Copilot coding agent, CLI-backed coding flows, background agents, cloud agents, or an SDK-hosted flow that adopts one of those schemas. Use platform for OS scope such as Windows, WSL/Linux, or macOS.

When the same specialist role needs different frontmatter, tool namespaces, or delegation mechanics across runtimes, create an agent variant: a runtime-specific .agent.md wrapper over shared instructions.

Official Runtime Facts to Design Around

Agent Variant Pattern

Split each specialized agent into two layers:

1. Shared instruction layer: persona, rules, workflow, artifacts, output contract, and tool-agnostic wording.
2. Runtime variant layer: target, tools, agents, handoffs, mcp-servers, and any runtime-specific tool references.

The shared layer should say what capability is required, not which runtime-specific tool name to call. Write "read the file", "run tests", or "delegate to the reviewer" rather than embedding a runtime-specific token.

Tool Assignment by Runtime Family

Use the official target values in the variant frontmatter: vscode or github-copilot.

Assignment Heuristics

1. Start from the minimum capability set needed for the role.
2. Assign tools per runtime, not per persona. The reviewer persona may be read-only in every runtime, but the concrete tool names still differ.
3. Put MCP server selection in the variant that can actually load it.
4. Keep body text free of runtime-specific tool names unless the file is explicitly runtime-locked.
5. Re-measure orchestration after tool changes. A variant that adds subagent access, shell execution, or MCP tools has changed its effective safety boundary.

Failure Modes to Audit

• A VS Code-only tool name in a github-copilot variant silently disappears.
• A shared instruction body mentions a tool token that only exists in one runtime.
• A runtime variant inherits an all-tools default because tools was omitted unintentionally.
• A subagent-only variant still allows model invocation because disable-model-invocation was not set.
• A team treats Windows, Linux, or macOS as the runtime boundary when the real difference is the Copilot host.

Prompt Engineering for Enforcement

Agent prompts in a harness serve a different purpose than general-purpose prompts. They are mechanical contracts, not creative guidance.

• Be exhaustive about rules, terse about explanation. Don't explain why a rule exists in the agent prompt — put that in docs. The prompt should list exactly what to check.
• Include remediation in the output format. When an observer reports a violation, it should include the fix instruction. This becomes context for the actor agent that fixes it.
• Avoid open-ended judgment calls. "Review code quality" is too vague. "Check that every public function in src/server/ has a corresponding test in tests/server/" is enforceable.
• Reference, don't inline. Agent prompts should point to ARCHITECTURE.md or docs/conventions/ for rule details rather than reproducing them. This prevents the prompt from going stale independently.

Strategy 3: Operational Audit Protocols

Harnesses usually fail in the operational details, not in the high-level design. Convert repo-specific review checklists into a reusable audit protocol with explicit passes and measurable outputs.

The Five Audit Passes

Audit Output Contract

Every audit pass should report findings in a fixed shape:

Compression Is a Harness Capability

Prompt compression is not cosmetic. It preserves budget for task context and reduces drift.

Always remove:

• Motivational preambles before the real contract
• Inline explanations of why a rule exists
• Duplicated protocol blocks repeated per mode
• Example output blocks that only restate an existing schema
• Verbose decision trees better expressed as flat steps

Always preserve:

• All numbered rules in the enforcement contract
• Every supported mode name and its complete step list
• Exact path patterns and naming conventions
• Contract field names, required/optional status, and signal schema
• Preflight gates and failure-mode handling

What to Measure

For every auditable harness, keep a small measurement set close at hand:

1. Prompt body length after frontmatter stripping.
2. Combined wrapper-plus-embedded body length for distributed agents.
3. Count of active signal types and whether all files use the same schema.
4. Artifact ownership table: one writer per artifact, explicit shared append-only logs.
5. Revalidation triggers: which edits force which audit passes.

Strategy 4: Entropy Management

Every agent-generated line of code can introduce drift. Entropy management is the discipline of detecting and correcting drift before it compounds.

Golden Principles Framework

Define a small set (5–10) of non-negotiable, mechanically verifiable rules:

Key insight: Golden principles must be verifiable by linters, structural tests, or agents — not just documented. If you can't automate the check, it's a guideline, not a golden principle.

Garbage Collection Cadence

Quality Scoring Pattern

Maintain a versioned QUALITY_SCORE.md that grades each domain and layer:

| Domain | Types | Config | Service | Tests | Docs | Overall |
|--------|-------|--------|---------|-------|------|---------|
| Auth   | A     | A      | B       | B     | C    | B       |
| Billing| A     | B      | B       | C     | D    | C+      |
| Search | B     | B      | C       | D     | F    | D+      |

This gives both humans and agents a map of where debt lives. A maintainer agent can read this and prioritize: "Search.Docs is F — generate missing documentation for the Search domain."

Entropy Detection Strategies

1. Pattern divergence scan. Compare how a pattern is implemented across modules. If 8 out of 10 services use structured logging but 2 use console.log, flag the outliers.
2. Doc/code freshness ratio. If src/billing/ changed 30 times this month but docs/billing.md changed 0 times, the docs are likely stale.
3. Lint error message engineering. Write custom lint error messages that include remediation instructions. When a linter reports "Import from auth/ in billing/ violates boundary — move shared types to shared/types/", the agent can act on it directly. The error message is the agent's instruction.
4. Snapshot diffing. Periodically generate a snapshot of the repo structure (file tree, import graph, export map) and diff against the previous snapshot. Large deltas without corresponding doc updates signal drift.

Strategy 5: Application Legibility

Agents can only verify what they can observe. Extending the agent's senses beyond static files dramatically increases harness leverage.

Legibility Layers

Legibility Best Practices

1. Make the app bootable per task. If an agent can't start the application in isolation, it can't validate its own work. Design for single-command startup (Docker Compose, Aspire, etc.).
2. Wire browser automation for UI verification. An agent that edits a Blazor component should be able to screenshot the result. This closes the feedback loop without human eyes.
3. Expose observability to the agent. If you have logs and metrics, make them queryable. An agent instruction like "ensure no error logs during startup" is only enforceable if the agent can read the logs.
4. Ephemeral environments. Each agent task should run against an isolated instance. Shared environments introduce cross-task interference that agents can't reason about.

Strategy 6: Harness Evolution

A harness is not a one-time setup. It evolves with the codebase.

Maturity Model

Evolution Workflow

1. Assess. Use the harness assessment from the harness-engineering skill. Rate each component 0–2.
2. Target one level up. Don't jump from Level 1 to Level 5. Move one level at a time per component.
3. Instrument before enforcing. Before adding a linter, add an observer agent to measure the current state. Quantify violations before blocking them.
4. Encode human taste continuously. Every time a human reviews agent output and says "that's not how we do it here," capture the rule — as a lint rule, instruction update, or golden principle. Never rely on the same correction twice.
5. Retroactively harness brownfield code. Start with context engineering (cheapest, highest ROI), add constraints per-module as you touch them, and add entropy management last.

Revalidation Triggers

Tie harness maintenance to concrete change events instead of vague periodic reviews.

If a harness cannot tell you which checks to rerun after a change, it is still too implicit.

Scaling Across Repositories

For organizations with multiple repos:

• Harness template repos. Create starter repo templates with pre-configured instruction files, standard agent fleet, and CI validation jobs. This parallels "golden path" service templates, optimized for agent-driven development.
• Org-level agents. Define organization-wide enforcement agents (via .github-private repo) that apply everywhere. Keep repo-level agents for domain-specific concerns.
• Shared skills as packages. Extract common skills (architecture validation, deployment runbooks) into a shared repo and publish to personal skill directories for cross-repo reuse.
• Federated golden principles. Maintain a core set of golden principles organization-wide, with repo-specific extensions. Enforce the core set from org-level; let repos own their extensions.

Anti-Patterns