# Harness Feedback

> Use when an agent says a test, VM, proof, evaluator, or release gate is overloaded, too strict, blocking staging, or causing false positives; split checks by profile, measure the burden, preserve high-risk evidence, and verify the smallest corrected workflow. Do not use for ordinary test selection, a single test failure, or a full security audit without a harness-scope question.

- Skill: `anastasiyaw/harness-feedback` (Agent Skill)
- Install (CLI): `npx skillmds@latest add anastasiyaw/harness-feedback`
- Raw SKILL.md: https://api.skillmd.com/api/skills/anastasiyaw/harness-feedback/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: AnastasiyaW (https://skillmd.com/u/anastasiyaw)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/anastasiyaw/harness-feedback

---


# Harness Feedback

Treat "the harness is too strict" as an engineering finding, not as permission
to disable a safety check. Find the boundary that owns the mismatch and move
the check to the narrowest profile that actually needs its evidence.

## Profiles

Use these profiles unless the project has a more specific, documented contract:

| Profile | Purpose | Typical blocking checks |
|---|---|---|
| `staging-smoke` | Fast proof that the changed build starts and the critical path works | build, focused regression, one stable smoke/contract check |
| `security-proof` | Prove an adversarial or trust-boundary claim | hostile tests, source/collector proof, fresh-context evaluator |
| `release-attestation` | Prove the exact releasable artifact and its identity | signing, Authenticode/tool identity, installer/package checks |
| `nightly-stress` | Find intermittent and capacity failures | race, stress, AV/OS matrix, long-running evals |

`staging-smoke` must not require signing, production credentials, a release
certificate, or a long VM stress run. `security-proof` may run on an unsigned
staging build when its claim is source or runtime behavior. A release check may
remain blocking for release promotion without becoming a per-edit gate.

## Feedback Loop

For every overload signal, record:

1. requested profile and change boundary;
2. gate that blocked or dominated the run;
3. command, elapsed time, failure count, and evidence actually produced;
4. whether the gate was relevant, duplicated, flaky, or misplaced;
5. the smallest profile split or deletion of duplicate coverage;
6. a before/after run of the affected profile and a fresh review of the rule.

Use the deterministic `harness-load-advisor.py` signal as an intake event. It
stores metadata outside the repository and forces the final report to name the
mismatch. Durable policy changes belong in Git; raw session traces do not.

## Required Report

Do not write "overkill" and move on. Report:

```text
Harness feedback: OVERLOAD | CLEAR
Requested profile: staging-smoke | security-proof | release-attestation | nightly-stress
Mis-scoped gate: <name>
Evidence: <command, result, elapsed time, or explicit missing proof>
Correction: <profile split or rule change>
Verification: <before/after commands and result>
Residual risk: <what remains intentionally gated and where>
```

## Gotchas

- A fresh evaluator is an independence control, not a release-signing check.
- A VM can be a reusable execution environment without forcing release identity
  checks into every VM smoke.
- A green fast gate does not prove release readiness; a red release-only gate
  does not invalidate a staging smoke unless the staging claim depends on it.
- Do not replace a misplaced gate with retries, sleeps, or a bypass marker.
- Do not infer overload from one slow run; distinguish environment failure from
  a profile contract error.

## Troubleshooting

| Symptom | Likely cause | Action |
|---|---|---|
| Staging smoke asks for signing | Release gate leaked into staging profile | split `release-attestation` and run the smoke on the unsigned staging artifact |
| Security proof blocks on a production VM | Runtime environment and release identity are coupled | keep the VM, remove release-only assertions from the security profile |
| Same gate fails repeatedly | Wrong scope, flaky boundary, or missing fixture | classify the failure and add a focused reproducer; never silently retry |
| Agent says "tests passed" with no profile | Evidence contract is incomplete | require the report fields above and the exact command/result |
| Fix removes a safety check | Causal ownership was not traced | restore the check, document the narrower boundary, and re-verify it there |