Engagement Retrospective
The close-out retro: after a completed engagement (or campaign, or
framework push), record where execution drifted from prescribed
discipline and what harness or skill change each drift produced.
The retro is the mechanism that turns repeated mistakes into standing
fixes; without it the same drift recurs on the next engagement.
Fires as the final link of the closeout chain
(CLOSEOUT_CHAINS in scripts/offensive.py: triage -> evidence ->
walkthrough -> THIS SKILL).
Wiki
Hub: [[methodology-moc]] - Primary page: [[multi-agent-campaign-orchestration]]
Anchors: [[bug-hunting-methodology]], [[vulnerability-reports]]
REFS: [[multi-agent-campaign-orchestration]]
Paths (binding - define once at the top of the run notes)
# seat side (WSL via /mnt/c; ported 2026-10-17) - engagement tree is
# DOUBLE-NESTED in VMshare:
ENG_DIR=/mnt/c/Users/Lenovo/Desktop/VMshare/<eng>/<eng>
# repo tree (admin/ledgers):
REPO=/mnt/c/Users/Lenovo/Documents/ObisidianVaults/ZTorch/ZTorch
# the retro log itself (generic, codenamed, never client-named):
session/retro-log.md (append; harness-side, shared across engagements)
Retro outputs NEVER land under targets// (client tree) - the retro
is harness knowledge, written generic.
When to Use
- An engagement, campaign, or multi-session task just completed
(success or failure)
- The user asks "what went wrong", "do a retro", "why did that take so
long"
- A recurring mistake was noticed mid-work and should be captured
before it is forgotten
Don't use for: single bugs with a clear root cause (systematic
debugging); code review of changes (ponytail-review).
Where the outputs land (one layer per output)
Route each output to the layer that will actually re-surface it:
| Output |
Land it in |
Why |
| A discipline fix for a recurring situation |
the SKILL itself via skill_manage (patch the skill whose mandate was missed) |
the fix loads exactly when the situation recurs |
| A cross-session fact (env quirk, seat config, model behavior) |
memory tool (Hermes GUI seat; wsl claude cli: project memory/CLAUDE.md) - one declarative entry |
auto-injected every session, no rereading needed |
| "Have we hit this before" questions |
session_search (Hermes GUI seat; wsl claude cli: transcript/session recall) |
transcript recall beats archaeology |
| A reusable offensive technique (generic, leak-checked) |
wiki via scripts/wiki-stage.py -> scripts/wiki-promote.py |
the only sanctioned path into the wiki |
| The human-readable audit trail |
session/retro-log.md (append, generic wording only) |
history stays greppable |
| A new mechanical check |
a script in scripts/ (falsifiable, exit 0 on pass, tested) |
code beats prose for mechanical gates |
Procedure
- Collect the evidence first. Read the session transcript, tool
call logs, state files, and any drift/telemetry markers BEFORE
writing anything. Do not reconstruct from memory. Completion
criterion: you can cite concrete tool calls or file states for
every claim.
- List each drift with a root cause, not a symptom. For each:
what was supposed to happen (which skill or rule), what actually
happened, and WHY the discipline failed (missing rule, rule ignored
under momentum, tooling gap, or environment failure). Name the
failure's owner honestly: agent drift, harness gap, or environment.
- Route each drift to exactly one action:
- Skill fix: the rule existed but was under-specified or in the
wrong place.
- New mechanism (hook/script/checklist item): the failure is
mechanically detectable - only if deterministic; judgement calls
stay as skill prose.
- Log-only negative knowledge: a rabbit hole worth recording so it
is never re-entered (Deadends.md pattern).
- No change: the reflex fired and was ignored; note it as operator
discipline.
- Also record what went right. Validated disciplines need
confirmation too, or a later cleanup pass deletes them as unused -
a past de-bloat deleted two capture reflexes a prior retro had
proven necessary, and the drifts recurred.
- Mark each fix DONE / PROPOSED / DEFERRED. Small fixes land this
session with tests or a skill edit; larger mechanisms go to the
operator as a proposal with a one-line rationale. Every retro
sweep updates the proposal registry
(
docs/framework-proposals.md): re-count each open row's
recurrences, add new proposals as rows. Any item reaching 2
recurrences gets scheduled; 3 forces it into the next improvement
pass.
- Write generic, never client-specific. Codename the engagement;
no hosts, IPs, creds, or client-identifying detail, so the retro
file is safe to keep and share. check-leaks before any push.
- Prefer the smallest fix that closes the class. A tightened
skill line beats a new hook; a new hook beats a new subsystem.
Complex failures get a root-cause pass before a fix: one retro
found the "root cause" was a regression from an earlier cleanup,
not missing prose.
Recurring patterns worth checking for
- Abandoned the prescribed driver/plan once manual work started
flowing (the meta-drift; the most expensive one in the source data)
- Grep-as-read on results that needed a full read (the load-bearing
line is what grep skips)
- Serial work that should have been a parallel fan-out
- Advisory nudges ignored under momentum; capture deferred to close-out
- Silent tooling failure read as a negative result (tool never ran vs
ran and found nothing)
- Re-deriving from memory what a reference page or skill already
documents
- Validator matched its own scaffold: a fill-detector regex whose
\S
can cross a newline matches the template's next section (an HTML
comment marker) and passes on a blank form. Anchor same-line checks
with [ \t]+\S, never \s*\S
- Evidence detector written against fixture shapes only: galleries use
named alt text (
), captures live in per-lane
subdirs - test detectors against a REAL completed artifact, not only
synthetic fixtures shaped like the detector expects
Verification
- Every drift in the retro names a root cause and an action with a
status.
- Every DONE fix has a concrete artifact: a skill edit, a test, or a
file change.
- No client-identifying data anywhere in the retro file.
- Close-out ordering: stamp the completion marker AFTER the last write
to the engagement's state/walkthrough files. Those gates re-arm when
the marker is older than state.md (staleness rule), so
marker-then-document ordering re-arms the nag even though the work
is done - if a hook nags about an already-done step, check the
detector's freshness rule and re-stamp instead of redoing work.
- Retro applies to framework-improvement sessions too, not only field
engagements: harness edits, gate fixes, and doctrine ports get a
retro-log entry with their test evidence, and any nag that misfired
gets its detector root-caused (a nag that is wrong about a completed
task is a detector bug, not operator drift).
Exit artifact
Appended session/retro-log.md entry (codename, drifts + root causes +
routed actions + statuses), plus whatever the routing table landed
(skill patches committed, memory entries saved, wiki candidates
staged). The closeout chain is DONE when the retro entry exists and
every PROPOSED item is parked with its owner.
Stop conditions
Retro entry appended, every PROPOSED item parked with an owner, no
unchecked boxes above REVIEW, nothing written into skill dirs
(skills/workflow/closeout/engagement-retro/ stays untouched). hunt-core
stop conditions fire unchanged for any field-engagement content the
retro touches.
1---2name: engagement-retro3description: "Use when an engagement, campaign, or framework push closes: extract drifts + fixes into routed actions (skill patches, memory entries, wiki staging, scripts). Turns repeated mistakes into standing fixes."4metadata:5 source: https://skillmd.com/skills/encod3d-sec/engagement-retro6 hermes:7 tags: [retrospective, closeout, drift-analysis, self-learning, torchv2]8 related_skills: [evidence, triage, walkthrough, wiki, ponytail-review]9---1011# Engagement Retrospective1213The close-out retro: after a completed engagement (or campaign, or14framework push), record where execution drifted from prescribed15discipline and what harness or skill change each drift produced.16The retro is the mechanism that turns repeated mistakes into standing17fixes; without it the same drift recurs on the next engagement.1819Fires as the final link of the closeout chain20(`CLOSEOUT_CHAINS` in scripts/offensive.py: triage -> evidence ->21walkthrough -> THIS SKILL).2223## Wiki2425Hub: [[methodology-moc]] - Primary page: [[multi-agent-campaign-orchestration]]26Anchors: [[bug-hunting-methodology]], [[vulnerability-reports]]27**REFS:** [[multi-agent-campaign-orchestration]]2829## Paths (binding - define once at the top of the run notes)3031 # seat side (WSL via /mnt/c; ported 2026-10-17) - engagement tree is32 # DOUBLE-NESTED in VMshare:33 ENG_DIR=/mnt/c/Users/Lenovo/Desktop/VMshare/<eng>/<eng>34 # repo tree (admin/ledgers):35 REPO=/mnt/c/Users/Lenovo/Documents/ObisidianVaults/ZTorch/ZTorch36 # the retro log itself (generic, codenamed, never client-named):37 session/retro-log.md (append; harness-side, shared across engagements)3839Retro outputs NEVER land under targets/<eng>/ (client tree) - the retro40is harness knowledge, written generic.4142## When to Use4344- An engagement, campaign, or multi-session task just completed45 (success or failure)46- The user asks "what went wrong", "do a retro", "why did that take so47 long"48- A recurring mistake was noticed mid-work and should be captured49 before it is forgotten5051Don't use for: single bugs with a clear root cause (systematic52debugging); code review of changes (ponytail-review).5354## Where the outputs land (one layer per output)5556Route each output to the layer that will actually re-surface it:5758| Output | Land it in | Why |59|---|---|---|60| A discipline fix for a recurring situation | the SKILL itself via `skill_manage` (patch the skill whose mandate was missed) | the fix loads exactly when the situation recurs |61| A cross-session fact (env quirk, seat config, model behavior) | `memory` tool (Hermes GUI seat; wsl claude cli: project memory/CLAUDE.md) - one declarative entry | auto-injected every session, no rereading needed |62| "Have we hit this before" questions | `session_search` (Hermes GUI seat; wsl claude cli: transcript/session recall) | transcript recall beats archaeology |63| A reusable offensive technique (generic, leak-checked) | wiki via `scripts/wiki-stage.py` -> `scripts/wiki-promote.py` | the only sanctioned path into the wiki |64| The human-readable audit trail | session/retro-log.md (append, generic wording only) | history stays greppable |65| A new mechanical check | a script in scripts/ (falsifiable, exit 0 on pass, tested) | code beats prose for mechanical gates |6667## Procedure68691. **Collect the evidence first.** Read the session transcript, tool70 call logs, state files, and any drift/telemetry markers BEFORE71 writing anything. Do not reconstruct from memory. Completion72 criterion: you can cite concrete tool calls or file states for73 every claim.742. **List each drift with a root cause, not a symptom.** For each:75 what was supposed to happen (which skill or rule), what actually76 happened, and WHY the discipline failed (missing rule, rule ignored77 under momentum, tooling gap, or environment failure). Name the78 failure's owner honestly: agent drift, harness gap, or environment.793. **Route each drift to exactly one action:**80 - Skill fix: the rule existed but was under-specified or in the81 wrong place.82 - New mechanism (hook/script/checklist item): the failure is83 mechanically detectable - only if deterministic; judgement calls84 stay as skill prose.85 - Log-only negative knowledge: a rabbit hole worth recording so it86 is never re-entered (Deadends.md pattern).87 - No change: the reflex fired and was ignored; note it as operator88 discipline.894. **Also record what went right.** Validated disciplines need90 confirmation too, or a later cleanup pass deletes them as unused -91 a past de-bloat deleted two capture reflexes a prior retro had92 proven necessary, and the drifts recurred.935. **Mark each fix DONE / PROPOSED / DEFERRED.** Small fixes land this94 session with tests or a skill edit; larger mechanisms go to the95 operator as a proposal with a one-line rationale. **Every retro96 sweep updates the proposal registry**97 (`docs/framework-proposals.md`): re-count each open row's98 recurrences, add new proposals as rows. Any item reaching 299 recurrences gets scheduled; 3 forces it into the next improvement100 pass.1016. **Write generic, never client-specific.** Codename the engagement;102 no hosts, IPs, creds, or client-identifying detail, so the retro103 file is safe to keep and share. check-leaks before any push.1047. **Prefer the smallest fix that closes the class.** A tightened105 skill line beats a new hook; a new hook beats a new subsystem.106 Complex failures get a root-cause pass before a fix: one retro107 found the "root cause" was a regression from an earlier cleanup,108 not missing prose.109110## Recurring patterns worth checking for111112- Abandoned the prescribed driver/plan once manual work started113 flowing (the meta-drift; the most expensive one in the source data)114- Grep-as-read on results that needed a full read (the load-bearing115 line is what grep skips)116- Serial work that should have been a parallel fan-out117- Advisory nudges ignored under momentum; capture deferred to close-out118- Silent tooling failure read as a negative result (tool never ran vs119 ran and found nothing)120- Re-deriving from memory what a reference page or skill already121 documents122- Validator matched its own scaffold: a fill-detector regex whose `\S`123 can cross a newline matches the template's next section (an HTML124 comment marker) and passes on a blank form. Anchor same-line checks125 with `[ \t]+\S`, never `\s*\S`126- Evidence detector written against fixture shapes only: galleries use127 named alt text (``), captures live in per-lane128 subdirs - test detectors against a REAL completed artifact, not only129 synthetic fixtures shaped like the detector expects130131## Verification132133- Every drift in the retro names a root cause and an action with a134 status.135- Every DONE fix has a concrete artifact: a skill edit, a test, or a136 file change.137- No client-identifying data anywhere in the retro file.138- Close-out ordering: stamp the completion marker AFTER the last write139 to the engagement's state/walkthrough files. Those gates re-arm when140 the marker is older than state.md (staleness rule), so141 marker-then-document ordering re-arms the nag even though the work142 is done - if a hook nags about an already-done step, check the143 detector's freshness rule and re-stamp instead of redoing work.144- Retro applies to framework-improvement sessions too, not only field145 engagements: harness edits, gate fixes, and doctrine ports get a146 retro-log entry with their test evidence, and any nag that misfired147 gets its detector root-caused (a nag that is wrong about a completed148 task is a detector bug, not operator drift).149150## Exit artifact151152Appended session/retro-log.md entry (codename, drifts + root causes +153routed actions + statuses), plus whatever the routing table landed154(skill patches committed, memory entries saved, wiki candidates155staged). The closeout chain is DONE when the retro entry exists and156every PROPOSED item is parked with its owner.157158## Stop conditions159160Retro entry appended, every PROPOSED item parked with an owner, no161unchecked boxes above REVIEW, nothing written into skill dirs162(skills/workflow/closeout/engagement-retro/ stays untouched). hunt-core163stop conditions fire unchanged for any field-engagement content the164retro touches.