Harness Feedback

Harness Feedback Loop

When friction is discovered during harness usage — a hook that didn't fire, guidance that was contradictory, a mandate violated in practice, or tool output diverging from documentation — capture and escalate it by default. The goal is to ensure harness improvements flow upstream without requiring users to notice and report them manually.

Procedure

1. Address the immediate problem

Work around or fix the specific friction blocking your session — do not defer this step or leave it for later. Examples:

• A hook expected to run but didn't? Inspect core.hooksPath, check hook file permissions, run it manually if necessary.
• Guidance was unclear? Disambiguate it by reading related docs or asking for clarification.
• A mandate was hard to apply? Document what made it hard and propose a better framing.

2. Log it locally

Append a dated entry to docs/operational/harness-feedback.md (create it with a short header if it doesn't exist). Use this template:

## [ISO 8601 date] – [one-line summary]

**Recurrence key:** `[stable-kebab-case-slug]`

**Harness version:** [commit SHA or tag of the harness in use]

**What happened:** [The friction event and when you noticed it]

**Root cause:** [Why did this happen? What in the harness design or docs caused it?]

**Impact:** [How did it affect your session or workflow?]

**What agentharness should change:** [Concrete recommendation — a new check, clearer guidance, a new tool, a code fix, etc.]

**Corrective action taken:** [What you did to work around or fix it] Logged upstream as #[issue-number] — or "Not filed upstream (opted out via .agentharness-no-upstream-feedback)".

Optional structured fields. Add any of these when you actually have the information — none are mandatory, and a one-off entry with just the fields above is still a complete, valid entry:

**Event class:** [hook-failure | ambiguous-guidance | mandate-violation | tool-output-mismatch | other]

**Observed vs. inferred:** [Directly observed (you saw the failure happen) or Inferred (you're reasoning from symptoms without a direct repro)]

**Evidence reference:** [Link or path to the transcript, log, or command output that shows this — not the raw content, a pointer to it]

**Severity:** [blocking | degraded | cosmetic]

**Workaround:** [What let the session continue despite the friction, if anything]

**Resolution:** [Fixed upstream in #N | Still open | Won't-fix, because ...]

These exist because 22 local entries in, recurrence detection was still a human remembering an earlier entry by feel — the same trunk-protection gap got filed twice two months apart before the recurrence-key field existed, and volume has since roughly doubled again. Add a field when it helps you or a future reader classify or search the entry; skip it when it wouldn't (most entries won't have a meaningful "evidence reference," for instance, and that's fine).

Corroboration before promotion. A feedback entry is a report, not authorization to change policy. Before a recurring theme turns into an actual mandate, guide edit, or new check, require at least one of: reproduction, a second independent occurrence (not the same session re-describing itself), or deterministic evidence (a log, a failing test, a hook's actual output) — not just an agent's own interpretation of what went wrong. This mirrors the Recommendation Assessment mandate's existing "assess before implement" step; corroboration is what earns a feedback entry the right to be assessed as a real recommendation rather than one session's read of events.

Not built by this schema. No JSON export, aggregation command, or audit-summary tooling — reading 22 markdown entries with grep still works, and building a report command ahead of a demonstrated aggregation need would be the exact kind of unscoped machinery this repo pushes back on. Retention/expiry for stale entries is explicitly out of scope here too — that's docs/operational/'s general forgetting-policy gap, tracked separately (issue #199, deferred pending its own evidence trigger), not something this skill should decide on its own. Revisit both if aggregation or retention friction actually shows up.

About the recurrence key and harness version. They exist because recurrence is the signal that turns a one-off annoyance into something worth changing, and recurrence was being caught only by a human happening to remember an earlier entry — the same trunk-protection gap was filed twice two months apart, and one root cause has been filed as both an issue and a comment on a different issue.

• Recurrence key — a stable slug for the underlying cause, not this symptom. Before writing a new entry, grep existing entries and upstream issues for the key you're about to use: if it already exists, you are looking at a recurrence, and say so in the entry and the upstream issue rather than filing a fresh unrelated-looking report. Two different symptoms of one cause should share a key (e.g. a lock expiring too early and a lock expiring too late both being agent-lock-liveness).
• Harness version — which harness the friction was observed against, so a report can be told apart from one already fixed since. git -C <harness-dir> rev-parse --short HEAD is enough.

3. File upstream

Unless .agentharness-no-upstream-feedback exists at this repo's root (a flag at the consuming repository, not the harness itself), file the same finding as a GitHub issue on andr-ca/agentharness. Use gh issue create -R andr-ca/agentharness with:

Title: Short, actionable summary (e.g., "Hook X doesn't fire on merge commits", "Mandate Y is ambiguous without threshold")

Body: Include these three structural elements in order:

1. Origin repo: Consuming project name and repo URL where friction was discovered
2. Full context: Not just the symptom, but the chain of events, what you tried, what failed. Match the detail bar set by exemplar issues #76 (hook gap + impact), #77 (mandate ambiguity + recurring failure), #78 (staleness detection gap).
3. Concrete recommendation: What specifically agentharness should change — a new script, a clarity edit, a schema addition, etc.

Record the issue number in the local entry's Corrective action taken field and link back to it. The GitHub issue should reference the origin repo and this entry so both directions are auditable.

Rules

• No ask-the-user step for logging or filing. The absence of .agentharness-no-upstream-feedback is standing authorization — do not prompt the operator when the flag is absent.
• Do this as part of handling the friction, not afterward. If you fix a hook failure, log it and file it before moving on to the next task. Memory is not a backup plan.
• Both directions always. Local entry → issue link; GitHub issue → origin repo reference and local entry. Bidirectional traceability.
• Redact before you write — locally and upstream. The upstream repo is public. Never include secrets, credentials, tokens, PII, customer data, or proprietary code in the local log or the issue; summarize sensitive logs rather than pasting them verbatim, and replace identifying values with placeholders. If the finding can't be described without sensitive detail, log locally with redactions and skip the upstream filing, noting why.
• Keep dates in ISO 8601 format (e.g., 2026-07-17) for sorting and CI matching.

Output shape

Issue filed? Update the calling session's context with: the issue number/URL, and the local file path where you recorded it. Both are now part of the session's findings — they don't need to be hidden from the operator.