Harness Optimize

Harness Optimize

End-to-end loop: research → inventory → proposed diffs (primary artifact) + scorecard (secondary) → STOP for approval → apply only approved ids → cache.

Trust principle (product)

A harness doctor is trustworthy when its own fixes are reviewable.

• The scorecard is useful context.
• The artifact to keep is each proposed fix: which config, which permission/setting, why now, exact diff or patch, how to revert.
• Prefer a short stack of reviewable proposals over a long prose lecture.

Hard gates (non-negotiable)

1. Propose before implement. Every run must present the findings report with a Proposed fixes section (reviewable diffs) before any config mutation. Scorecard supports prioritization; it does not replace proposals.
2. Never change without permission. Do not edit configs until the user approves specific recommendation ids via ask_user_question. Goal mode, yolo, and always-approve do not count as apply permission.
3. Use the question tool for user decisions (scope, which proposals to apply).
4. Default mode is report-only. After proposals, always ask what to apply.

Allowed without asking: read files, run diagnostic scripts under scripts/ (discover, inventory, mcp-doctor, env-check, mcp-matrix, dual-rules, permission-audit, recurrence-check, restore-kit --list, run-diagnostics), grok mcp doctor / version commands, web fetch of changelogs, write report text into the chat (or a path the user already named). Forbidden without prior explicit apply approval: cp/mv/rm of harness configs, editing config files, enabling/disabling MCP, moving secrets, archiving agents/skills, rewriting CLAUDE.md/AGENTS.md.

Modes

apply in $ARGUMENTS means “user is open to applying after the report” — it does not skip the report or the permission question.

State file (create dir if needed; write state.json only for report metadata after the report, or for apply results after approved apply):

$HARNESS_OPTIMIZE_STATE (default under skill or GROK_PLUGIN_DATA)

Phase 0 — Target + safety

1. Detect which harness is running this session (env, binary, skill path).
2. If target harness is ambiguous and not in $ARGUMENTS, use ask_user_question (multi-select OK). Do not invent scope.
3. State constraints:
   • No secret values in reports (redact tokens, client secrets, Bearer headers).
   • No force-push, no deleting without archive path and apply approval.
   • Respect one-in-one-out for ~/.claude surface when proposing adds.
   • Project edits: if git-tracked, prefer worktree or explicit commit; don't dirty shared checkouts silently.

Phase 1 — Release check + full changelog since last run

Do this before judging the install. Stale advice is worse than no advice.

1a. New Grok release available?

bash "$HO_ROOT/scripts/release-check.sh"
# or: grok update --check --json

If update_available | true:

• Add a High (or Critical if breaking channel) recommendation: upgrade Grok (grok update / pin version). Do not run the install without user approval.
• Still complete the rest of the report on the currently installed version.
• After the user upgrades, they should re-run /harness-optimize so the new changelog is fully catalogued.

1b. Catalog all changelog changes since last run

bash "$HO_ROOT/scripts/changelog-delta.sh"

This compares ~/.grok/CHANGELOG.md + CHANGELOG.json to:

• state.json → last_changelog_cursor.grok
• state.json → last_changelog_content_hash

Required behavior:

1. Include every release newer than the cursor (not only the latest).
2. If the content hash changed for the same version, re-list all bullets for that version (nothing dropped).
3. For each bullet, print it as informational, actionable, or breaking.
4. For every actionable/breaking bullet, add a concrete harness suggestion (rec_seed / report rec id). Do not stop at “no config keywords” without listing informational items — the user must see the full catalog.
5. Prefer local files; only then fetch remote docs if files are missing.

Also load docs as needed:

• Grok: ~/.grok/docs/user-guide/ (config, skills, hooks, MCP, permissions)
• Other harnesses: per references/harness-registry.md

1c. Other harnesses (if targeted)

Claude / Codex / etc.: same idea — version + changelog since cursor when those paths exist.

Phase 2 — System inventory + Grok diagnostics pack

Skill root (scripts live next to this file):

# Prefer plugin root when Grok provides it
if [ -n "${GROK_PLUGIN_ROOT:-}" ] && [ -d "$GROK_PLUGIN_ROOT/skills/harness-optimize/scripts" ]; then
  HO_ROOT="$GROK_PLUGIN_ROOT/skills/harness-optimize"
elif [ -d "$HOME/.grok/skills/harness-optimize/scripts" ]; then
  HO_ROOT="$HOME/.grok/skills/harness-optimize"
else
  # Installed-plugins hash dir varies; locate by SKILL.md sibling if agent knows path
  HO_ROOT="$(dirname "$(readlink -f "${BASH_SOURCE[0]:-$0}")" 2>/dev/null || true)"
fi
# State/reports: writable plugin data when available
export HARNESS_OPTIMIZE_STATE="${HARNESS_OPTIMIZE_STATE:-${GROK_PLUGIN_DATA:-$HOME/.grok/skills/harness-optimize}/state.json}"
export HARNESS_OPTIMIZE_REPORTS="${HARNESS_OPTIMIZE_REPORTS:-${GROK_PLUGIN_DATA:-$HOME/.grok/skills/harness-optimize}/reports}"

Prefer the pack (read-only):

bash "$HO_ROOT/scripts/run-diagnostics.sh"

Or individual scripts under $HO_ROOT/scripts/:

Capture (do not dump secrets):

Claude-specific token traps (verify with live facts, not folklore):

• ~/.claude/rules/**/*.md often loads recursively every prompt
• Active agents/*.md often load every prompt (archive unused)
• Skills are usually discovery-cheap until activated

Phase 3 — Per-project inventory

bash "$HO_ROOT/scripts/inventory.sh" --scope projects \
  --root "${PROJECTS_ROOT:-$HOME/projects}"
bash "$HO_ROOT/scripts/dual-rules.sh"

Per project folder, record presence and rough size of:

• CLAUDE.md, AGENTS.md, .claude/, .grok/, .codex/, .cursor/, .agents/, .mcp.json
• Project hooks/settings, skills, agents
• Package manager / stack signals (for MCP and hook recommendations)
• Tier if known from workspace map (factory / product / experiment) — quality gates should match tier

Flag:

• Dual CLAUDE.md + AGENTS.md with large overlap (Grok may load both) — use dual-rules.sh severity
• Project MCP overlapping broken global MCP
• Factory projects missing typecheck/test/hooks
• Experiment projects with full enterprise harness bloat

Phase 4 — Blindspot pass (unknown unknowns)

Before interviewing, list 5–10 items as RESOLVED or OPEN. Seed from references/blindspots.md. Typical OPEN categories:

• Primary daily harness vs secondary (configs should bias primary)
• Solo vs multi-agent vs AFK loops
• Risk posture (yolo / always-approve vs on-request)
• Token budget sensitivity
• Which projects are "active money" vs archive-class
• Whether plugins/MCP auth failures are acceptable noise
• Team-shared project configs vs personal-only

Phase 5 — Interview (architecture-affecting only)

Read references/interview-tree.md. Rules:

1. Always use ask_user_question for each needed decision (or one multi-select block of tightly related options). Never invent answers silently.
2. Prefer options + (Recommended) default over free text.
3. Skip questions already answered in state.json.preferences only if the user previously confirmed them in a prior interactive run (not agent-assumed).
4. Stop when remaining questions would not change recommendations.
5. If the user explicitly says “use recommended defaults” via the question tool or chat, record an assumption ledger and continue to the report — still do not apply changes without a separate apply approval.

Priority order (stop early if answers saturate):

1. Primary harness + secondary
2. Work mode (interactive craft / parallel agents / AFK loops / CI headless)
3. Permission posture
4. Context budget (lean / balanced / kitchen-sink)
5. Active project shortlist (or "all under ~/projects")
6. Cross-harness parity desire (symlink skills vs independent copies)

Do not ask “apply now?” in Phase 5 — that happens only after Phase 6.

Phase 6 — Proposed fixes (primary) + scorecard (secondary)

Primary deliverable: a stack of reviewable proposals (P1/C1/H1/…).
Secondary: scorecard for prioritization. Details: references/scoring.md, references/proposal-format.md.

Proposal contract (required for every fix recommendation)

Each id must include all of:

If you cannot produce a reviewable diff for a finding, label it observation only (no apply id) — do not invent an un-reviewable “just fix it” action.

Optionally persist proposals to:

$HARNESS_OPTIMIZE_REPORTS/proposals-YYYYMMDD.md

Scorecard (supporting)

Score 0–10 each (see references/scoring.md). Useful; not a substitute for diffs.

Report template

# Harness Optimize Report — YYYY-MM-DD

## Scope
Harnesses: … | Mode: full|delta | Primary: …

## Release status
- current / latest / update_available
- If update available: proposal id with exact `grok update` command + revert (n/a)

## Changelog catalog (since last run)
- Every release + every bullet (informational | actionable | breaking)

## Diagnostics summary
- Env / permission / MCP / dual-rules (short; details in scripts)

## Scorecard (secondary)
| Dimension | Score | Top finding |
|-----------|-------|-------------|

## Proposed fixes (primary artifact — review before apply)

### C1 — <title>
- **Priority:** Critical
- **Target:** `~/.grok/config.toml` → `[mcp_servers.xapi]`
- **Why now:** <doctor/changelog/inventory evidence>
- **Proposed change:**
  ```diff
  - enabled = true
  + enabled = false

• How to revert: restore ~/.grok/config.toml.bak.<ts> or set enabled = true
• Verify: bash …/mcp-doctor.sh / xapi-health.sh

H1 — …

…

Observations (no apply)

• Items without a safe reviewable patch

Restore kit

• Existing backups (paths)

Assumption ledger

• A1: …

Next step

No config changes have been made. Select proposal ids to apply, or report only.

Recommendation rules:

- Prefer **minimal reversible** diffs with measurable effect.
- Map each proposal to **evidence** (changelog, doctor, inventory).
- Cap volume: top **3 Critical + 5 High** unless user wants full dump.
- Never put secret values in diffs; use `${ENV}` / redaction.
- Failing MCP → disable/re-auth proposal with exact toml hunk.
- Dual rules → pointer/merge proposal with exact file paths.

### Phase 6b — Permission gate (required)

After **Proposed fixes** are shown, call **`ask_user_question`**:

- **Question:** Which **proposals** should I apply?
- **Options:** Report only / stop (Recommended) · Apply all Critical ·
  Apply Critical + High · Selected ids (Other) · Custom…

**STOP.** Do not enter Phase 7 without explicit ids.

## Phase 7 — Apply (only after Phase 6b approval)

Only the **approved** proposal ids — re-show each proposal’s diff, then:

1. Confirm the same **Proposed change** still applies (re-read target file).
2. Backup: `cp -a <target> <target>.bak.$(date +%Y%m%d-%H%M%S)`.
3. Apply the reviewed hunk only (no drive-by edits).
4. Verify per proposal’s **Verify** field.
5. Record applied ids + backup paths in `state.json`.
6. Final summary: **what changed / backup path / how to revert** for each id.

If the file drifted so the proposed diff no longer applies: **stop**, re-propose,
ask again — do not force a fuzzy patch.

## Phase 8 — Persist + recurrence

After the **report** (even if nothing applied / report-only), **always** update
changelog review markers so the next run only sees *new* changes:

From `changelog-delta.sh` output:

| State field | Set to |
|-------------|--------|
| `last_changelog_cursor.grok` | `suggest_cursor` |
| `last_changelog_content_hash` | `suggest_content_hash` |
| `last_changelog_reviewed_at` | current UTC ISO-8601 |
| `versions.grok` | current short version (e.g. `0.2.104`) |
| `last_release_check` | `{ current, latest, update_available, checked_at }` from release-check |

Also write evaluation metadata, scores, preferences the user confirmed. After
**apply**, append `applied` ids.

```json
{
  "evaluated_at": "ISO-8601 UTC",
  "mode": "full|delta",
  "harnesses": ["grok"],
  "versions": { "grok": "0.2.104" },
  "last_changelog_cursor": { "grok": "0.2.104" },
  "last_changelog_content_hash": "sha256…",
  "last_changelog_reviewed_at": "ISO-8601 UTC",
  "last_release_check": {
    "current": "0.2.104",
    "latest": "0.2.104",
    "update_available": false,
    "channel": "alpha",
    "checked_at": "ISO-8601 UTC"
  },
  "preferences": {},
  "scores": {},
  "open_items": [],
  "applied": [],
  "project_roots": ["~/projects"]
}

Suggest recurrence (do not schedule without asking via question tool):

• When update_available: upgrade then re-run /harness-optimize
• After major CLI upgrade: /harness-optimize delta
• Monthly: full
• Wire into /loop / loopy only if the user approves

Scripts

Prefer scripts for counting and path discovery; use model judgment for interview, scoring, and rec text. See references/features.md and recipes/README.md.

Related tools (compose)

Anti-patterns

• Implementing before showing the report (including “quick Critical fixes”)
• Assuming apply permission from yolo, goals, or “optimize everything”
• Skipping ask_user_question and treating silence or free-text as consent
• Judging setup from memory of old defaults without reading this week's changelog
• Archiving agents/skills without backup + explicit confirm
• "Optimize" by adding more always-on context
• Same kitchen-sink hooks on experiment toys and factory SaaS
• Averaging contradictory multi-harness patterns instead of picking a primary
• Printing MCP secrets or auth tokens into the report

When to load references

Install note (Grok-first)

Preferred: native plugin

grok plugin install <github-user>/harness-optimize --trust
grok plugin enable harness-optimize

Then /harness-optimize or natural language. State/reports prefer GROK_PLUGIN_DATA.

Do not install into Claude's skill tree unless the user asks.