extend-rmagent-so — Auto-extend the Security Observatory menu
The self-improvement loop for instance #10. The deterministic half
(gap_extend.py) runs hourly via launchd and handles Type A trigger
additions automatically. This skill is the JUDGMENT half: review what the
script did, build Type B probes, verify, and ship — without weakening a
single rmagent rule.
When to use
- After gated questions accumulate in
menu_gaps.jsonl
- When asked: "extend the rmagent menu", "process the observatory gaps",
"extend rmagent-so"
- After a soak or demo reveals phrasings the menu can't anchor
Paths
INSTANCE_DIR = ~/neuralos-instances/rmagent-so
INSTANCE_PY = $INSTANCE_DIR/instance.py
BRIDGE_PY = $INSTANCE_DIR/bridge.py
MENU_JSON = $INSTANCE_DIR/needle_menu.json
GAP_LOG = $INSTANCE_DIR/menu_gaps.jsonl
TYPE_B_LOG = $INSTANCE_DIR/menu_gaps_type_b.jsonl
STATE = ~/.rmagent/ (cases/, silent.json, canary_state.json,
needle-drift/, actuate-journal.jsonl)
ESTATE = /Users/olu/estate.yaml (override: RMAgent_ESTATE env)
ENGINE = ~/.agents/skills/rmagent-so/scripts (lib.py; engine via
~/.agents/skills/rmagent-core/scripts/loader.py)
EXPORT = ~/.claude/skills/chinook-app-blueprint/scripts/export_menu.py
VERIFY_TRUTH = ~/.claude/skills/chinook-app-blueprint/scripts/verify_truth.py
VERIFY_SELECT = ~/.claude/skills/chinook-app-blueprint/scripts/verify_selection.py
GAP_EXTEND = ~/.claude/skills/chinook-app-blueprint/scripts/gap_extend.py
PYTHON = /Library/Frameworks/Python.framework/Versions/3.12/bin/python3.12
SERVICE = ng.neuralos.rmagentso (port 8892, launchd KeepAlive)
The constitution (read first — nothing below overrides it)
- WATCH ONLY. No probe may wrap actuate, systemctl, usermod, firewall
edits — or anything outside
engine.ALLOWED (19 named questions). The
bridge refuses non-allowlisted asks before a packet is sent; keep it so.
- Credentials come from env /
~/.rmagent/creds.json / the scrt
store — never baked into bridge.py, never in the inventory, never in
this skill.
- Capped answers, errors as data, holes are answers. A new probe
returns
{"error": ...} rather than guessing; ROW_CAP holds.
- Prefer FAST over LIVE. A new probe should expose data that ALREADY
exists in
~/.rmagent state (case answers, silent ledger, canary
state, drift baselines) before it proposes a live door pull. Live
probes are slow and knock on real boxes.
The algorithm
Step 1 — Read the gap log
cat $GAP_LOG — JSON lines with question, guessed_tool,
confidence. Dedupe by question (keep newest). Empty → report and stop.
Also check $TYPE_B_LOG for anything queued by earlier runs.
Step 2 — Run the deterministic classifier (or review its output)
$PYTHON $GAP_EXTEND --instance-dir $INSTANCE_DIR --dry-run
- Type C (pre-routed families, <3 words, no guess, false gaps) → skip.
- Type A (≥2 content-word overlap with the guessed probe's triggers) →
the hourly learner usually already applied these; verify idempotence.
The trigger content-word ledger lives in the instance.py docstring —
keep it accurate when you add triggers.
- Type B → the real work. Apply with
--restart after verification:
$PYTHON $GAP_EXTEND --instance-dir $INSTANCE_DIR \
--restart "launchctl kickstart -k gui/$(id -u)/$SERVICE"
Step 3 — Design the Type B probe (the judgment call)
Study bridge.py first. Every probe follows the house shape: reads state
through the helpers (_answers(), _silent(), _correlation(),
_drift_keys(), _witness(), _j()), returns _meta(rows, extras) or
_err(message), caps at ROW_CAP, rides validation counters.
Before writing anything, ask in order:
- Does the data already exist in state? Check
~/.rmagent/cases/answers/*.json keys (witness__skill), CASE.md,
correlation.json findings, silent.json, canary_state.json,
needle-drift baselines. Most good probes are views over these.
- Is it a comparison or aggregate of existing probes? Follow
so_compare_health (double live attest + verdicts).
- Only then consider a new live pull — and prefer reusing
bridge.so_ask_live / so_attest_live instead of new transport code.
Precedents to copy:
so_logons — a view over cached edges answers + greedy-grounding
recovery (multi-word witness args degrade to estate-wide)
so_changes — boot-time comparison + needle-drift scoring via the
needle_drift.py CLI (subprocess, score key with the answer JSON on
stdin)
so_compare_health — double so_attest_live + verdicts
Step 4 — Probe hygiene (non-negotiable)
- Triggers: 3–8 lowercase phrases, token-disjoint against the LEDGER in
instance.py's docstring. Update the ledger.
- NEVER-docstring: positive statement + explicit rivals ("NEVER for the
question log (that is ask_log)").
- Arguments: prefer
str with bridge-side validation over giant Literal
enums — a 19-member Literal exceeded the engine's call token budget
("tool call truncated"). Enums ≤ 8.
- Grounding recovery: strip/handle greedy args (whole sentences stuffed
into a parameter — degrade to estate-wide like
so_logons; swap
obviously exchanged args like so_ask_live).
- High-confidence wrong probes are the #1 threat: a 0.98 fuzzy guess
passes the gate. If the soak shows a family being stolen by the
so_witness_detail magnet, give the family its own probe — that is the
cure, not more gate tuning.
- Record-ask quirk:
engine.record_ask writes the answer file EVEN ON A
HOLE — never let a battery/hole flow overwrite a good cached answer
(battery.py already guards this; keep the guard).
Step 5 — Export, verify, restart
$PYTHON $EXPORT $INSTANCE_DIR
$PYTHON $VERIFY_TRUTH --instance-dir $INSTANCE_DIR --cases $INSTANCE_DIR/truth_cases.jsonl
$PYTHON $VERIFY_SELECT --instance-dir $INSTANCE_DIR --cases $INSTANCE_DIR/selection_cases.jsonl
- Add at least one truth case per new probe (stable paths only) and one
selection case per trigger family (canonical + adversarial).
- LIVE probes: selection tests are route-only (no network). Smoke a real
ask once, manually — a silent host must return a hole, not an error page.
- Restart and verify:
launchctl kickstart -k gui/$(id -u)/$SERVICE && sleep 12
curl -s localhost:8892/menu # probe count must match the export
Step 6 — Archive + report
mv $GAP_LOG $GAP_LOG.$(date +%Y%m%dT%H%M%S).done
mv $TYPE_B_LOG $TYPE_B_LOG.$(date +%Y%m%dT%H%M%S).done # if non-empty
Report: gaps in / Type A applied / Type B built (name + one-line purpose)
/ verification numbers / new probe count / anything left gated as
by-design (out-of-domain fuel stays in the log on purpose).
Domain facts (current menu = 15 probes)
- Estate: ws1 (endpoint) + ws2 (data), PSRP doors, canaries
honeyadmin/svcbackup2, tracked Administrator/SYSTEM
- Two tiers: FAST (overview, witnesses, detail, findings, ask log, logons,
holes, canaries, drift, attackmap, allowlist, changes) and LIVE
(ask_live, attest_live, compare_health)
- Known holes: attackmap (pypsrp stream_error — rmagent-core fix needed)
and deepwindow (ETW payload empty post-boot); a fresh boot + 2 h window
legitimately shows 0 tracked logons
- Known gaps handled: kerberoasting/DC questions → so_allowlist (krb is
DC-only, not advertised); "flying blind" → so_holes; "lateral movement"
→ so_findings; "how about ws2" → so_witness_detail trigger (Type A,
applied autonomously by the learner)
Example run
$ extend-rmagent-so
menu_gaps.jsonl: 3 unique gaps
A: "show me the boxes" -> so_witnesses (applied by learner already — verified)
B: "show open ports" -> no state source; edges.conns only has tracked
outbound — queued as Type B, needs a hunt first
C: "hello" -> skip
Built: none this run (Type B queued)
TRUTH 25/25 · SELECTION 17/17 · menu 15 probes · service restarted
Safety rules (the short list)
- Never remove triggers; never modify existing bridge functions — add.
- Never touch CONF_GATE, the gate's two-signal rule, or
_trigger_hit.
- Never add a probe outside the watch-only constitution.
- Never put credentials in bridge.py, instance.py, or this skill.
- Test state-reading code against the LIVE
~/.rmagent state before
shipping — the shapes drift (Rev 17–21).
- Archive everything; the
*.done files are the loop's audit trail.
1---2name: extend-rmagent-so3description: >4 Automatically extend the rmagent-so neuralOS estate instance by reading5 menu_gaps.jsonl (the fuzzy-guess gate log), classifying each gap,6 applying Type A trigger fixes (script), building Type B probes (bridge +7 menu, over rmagent state that already exists), re-exporting8 needle_menu.json, verifying selection + truth, and restarting the9 observatory app on :8892. Run on-demand ("extend the rmagent menu") or10 after soaks. WATCH-ONLY: new probes must never wrap actuate or anything11 outside engine.ALLOWED.12metadata:13 source: https://skillmd.com/skills/drolu/extend-rmagent-so14---1516# extend-rmagent-so — Auto-extend the Security Observatory menu1718The self-improvement loop for instance #10. The deterministic half19(`gap_extend.py`) runs hourly via launchd and handles Type A trigger20additions automatically. This skill is the JUDGMENT half: review what the21script did, build Type B probes, verify, and ship — without weakening a22single rmagent rule.2324## When to use2526- After gated questions accumulate in `menu_gaps.jsonl`27- When asked: "extend the rmagent menu", "process the observatory gaps",28 "extend rmagent-so"29- After a soak or demo reveals phrasings the menu can't anchor3031## Paths3233```34INSTANCE_DIR = ~/neuralos-instances/rmagent-so35INSTANCE_PY = $INSTANCE_DIR/instance.py36BRIDGE_PY = $INSTANCE_DIR/bridge.py37MENU_JSON = $INSTANCE_DIR/needle_menu.json38GAP_LOG = $INSTANCE_DIR/menu_gaps.jsonl39TYPE_B_LOG = $INSTANCE_DIR/menu_gaps_type_b.jsonl40STATE = ~/.rmagent/ (cases/, silent.json, canary_state.json,41 needle-drift/, actuate-journal.jsonl)42ESTATE = /Users/olu/estate.yaml (override: RMAgent_ESTATE env)43ENGINE = ~/.agents/skills/rmagent-so/scripts (lib.py; engine via44 ~/.agents/skills/rmagent-core/scripts/loader.py)45EXPORT = ~/.claude/skills/chinook-app-blueprint/scripts/export_menu.py46VERIFY_TRUTH = ~/.claude/skills/chinook-app-blueprint/scripts/verify_truth.py47VERIFY_SELECT = ~/.claude/skills/chinook-app-blueprint/scripts/verify_selection.py48GAP_EXTEND = ~/.claude/skills/chinook-app-blueprint/scripts/gap_extend.py49PYTHON = /Library/Frameworks/Python.framework/Versions/3.12/bin/python3.1250SERVICE = ng.neuralos.rmagentso (port 8892, launchd KeepAlive)51```5253## The constitution (read first — nothing below overrides it)54551. **WATCH ONLY.** No probe may wrap actuate, systemctl, usermod, firewall56 edits — or anything outside `engine.ALLOWED` (19 named questions). The57 bridge refuses non-allowlisted asks before a packet is sent; keep it so.582. **Credentials** come from env / `~/.rmagent/creds.json` / the scrt59 store — never baked into bridge.py, never in the inventory, never in60 this skill.613. **Capped answers, errors as data, holes are answers.** A new probe62 returns `{"error": ...}` rather than guessing; ROW_CAP holds.634. **Prefer FAST over LIVE.** A new probe should expose data that ALREADY64 exists in `~/.rmagent` state (case answers, silent ledger, canary65 state, drift baselines) before it proposes a live door pull. Live66 probes are slow and knock on real boxes.6768## The algorithm6970### Step 1 — Read the gap log7172`cat $GAP_LOG` — JSON lines with `question`, `guessed_tool`,73`confidence`. Dedupe by question (keep newest). Empty → report and stop.74Also check `$TYPE_B_LOG` for anything queued by earlier runs.7576### Step 2 — Run the deterministic classifier (or review its output)7778```bash79$PYTHON $GAP_EXTEND --instance-dir $INSTANCE_DIR --dry-run80```8182- **Type C** (pre-routed families, <3 words, no guess, false gaps) → skip.83- **Type A** (≥2 content-word overlap with the guessed probe's triggers) →84 the hourly learner usually already applied these; verify idempotence.85 The trigger content-word ledger lives in the instance.py docstring —86 keep it accurate when you add triggers.87- **Type B** → the real work. Apply with `--restart` after verification:8889```bash90$PYTHON $GAP_EXTEND --instance-dir $INSTANCE_DIR \91 --restart "launchctl kickstart -k gui/$(id -u)/$SERVICE"92```9394### Step 3 — Design the Type B probe (the judgment call)9596Study `bridge.py` first. Every probe follows the house shape: reads state97through the helpers (`_answers()`, `_silent()`, `_correlation()`,98`_drift_keys()`, `_witness()`, `_j()`), returns `_meta(rows, extras)` or99`_err(message)`, caps at ROW_CAP, rides validation counters.100101Before writing anything, ask in order:1021031. **Does the data already exist in state?** Check104 `~/.rmagent/cases/answers/*.json` keys (witness__skill), CASE.md,105 correlation.json findings, silent.json, canary_state.json,106 needle-drift baselines. Most good probes are views over these.1072. **Is it a comparison or aggregate of existing probes?** Follow108 `so_compare_health` (double live attest + verdicts).1093. **Only then** consider a new live pull — and prefer reusing110 `bridge.so_ask_live` / `so_attest_live` instead of new transport code.111112Precedents to copy:113- `so_logons` — a view over cached edges answers + greedy-grounding114 recovery (multi-word witness args degrade to estate-wide)115- `so_changes` — boot-time comparison + needle-drift scoring via the116 `needle_drift.py` CLI (subprocess, score key with the answer JSON on117 stdin)118- `so_compare_health` — double `so_attest_live` + verdicts119120### Step 4 — Probe hygiene (non-negotiable)121122- Triggers: 3–8 lowercase phrases, token-disjoint against the LEDGER in123 instance.py's docstring. Update the ledger.124- NEVER-docstring: positive statement + explicit rivals ("NEVER for the125 question log (that is ask_log)").126- Arguments: prefer `str` with bridge-side validation over giant Literal127 enums — a 19-member Literal exceeded the engine's call token budget128 ("tool call truncated"). Enums ≤ 8.129- Grounding recovery: strip/handle greedy args (whole sentences stuffed130 into a parameter — degrade to estate-wide like `so_logons`; swap131 obviously exchanged args like `so_ask_live`).132- High-confidence wrong probes are the #1 threat: a 0.98 fuzzy guess133 passes the gate. If the soak shows a family being stolen by the134 so_witness_detail magnet, give the family its own probe — that is the135 cure, not more gate tuning.136- Record-ask quirk: `engine.record_ask` writes the answer file EVEN ON A137 HOLE — never let a battery/hole flow overwrite a good cached answer138 (battery.py already guards this; keep the guard).139140### Step 5 — Export, verify, restart141142```bash143$PYTHON $EXPORT $INSTANCE_DIR144$PYTHON $VERIFY_TRUTH --instance-dir $INSTANCE_DIR --cases $INSTANCE_DIR/truth_cases.jsonl145$PYTHON $VERIFY_SELECT --instance-dir $INSTANCE_DIR --cases $INSTANCE_DIR/selection_cases.jsonl146```147148- Add at least one truth case per new probe (stable paths only) and one149 selection case per trigger family (canonical + adversarial).150- LIVE probes: selection tests are route-only (no network). Smoke a real151 ask once, manually — a silent host must return a hole, not an error page.152- Restart and verify:153154```bash155launchctl kickstart -k gui/$(id -u)/$SERVICE && sleep 12156curl -s localhost:8892/menu # probe count must match the export157```158159### Step 6 — Archive + report160161```bash162mv $GAP_LOG $GAP_LOG.$(date +%Y%m%dT%H%M%S).done163mv $TYPE_B_LOG $TYPE_B_LOG.$(date +%Y%m%dT%H%M%S).done # if non-empty164```165166Report: gaps in / Type A applied / Type B built (name + one-line purpose)167/ verification numbers / new probe count / anything left gated as168by-design (out-of-domain fuel stays in the log on purpose).169170## Domain facts (current menu = 15 probes)171172- Estate: ws1 (endpoint) + ws2 (data), PSRP doors, canaries173 `honeyadmin`/`svcbackup2`, tracked `Administrator`/`SYSTEM`174- Two tiers: FAST (overview, witnesses, detail, findings, ask log, logons,175 holes, canaries, drift, attackmap, allowlist, changes) and LIVE176 (ask_live, attest_live, compare_health)177- Known holes: attackmap (pypsrp stream_error — rmagent-core fix needed)178 and deepwindow (ETW payload empty post-boot); a fresh boot + 2 h window179 legitimately shows 0 tracked logons180- Known gaps handled: kerberoasting/DC questions → so_allowlist (krb is181 DC-only, not advertised); "flying blind" → so_holes; "lateral movement"182 → so_findings; "how about ws2" → so_witness_detail trigger (Type A,183 applied autonomously by the learner)184185## Example run186187```188$ extend-rmagent-so189190menu_gaps.jsonl: 3 unique gaps191 A: "show me the boxes" -> so_witnesses (applied by learner already — verified)192 B: "show open ports" -> no state source; edges.conns only has tracked193 outbound — queued as Type B, needs a hunt first194 C: "hello" -> skip195Built: none this run (Type B queued)196TRUTH 25/25 · SELECTION 17/17 · menu 15 probes · service restarted197```198199## Safety rules (the short list)200201- Never remove triggers; never modify existing bridge functions — add.202- Never touch CONF_GATE, the gate's two-signal rule, or `_trigger_hit`.203- Never add a probe outside the watch-only constitution.204- Never put credentials in bridge.py, instance.py, or this skill.205- Test state-reading code against the LIVE `~/.rmagent` state before206 shipping — the shapes drift (Rev 17–21).207- Archive everything; the `*.done` files are the loop's audit trail.