RMAgent Red-Team (drill)
A purple-team drill that stages living-off-the-land artifacts on your Windows estate (WS1/WS2), then runs the rmagent-windows skill to score what it detects — and Telegrams you the report. This is how you answer "does rmagent actually work?"
It is a DRILL, not a real attack. Every artifact is prefixed RMAgentDrill_, uses benign payloads (a network test to 1.1.1.1, an echo to a temp file), is non-persistent, and is fully reversible with clean. It requires --confirm and never runs against a box you do not administer. Misses that are witness_blind are holes, not so bugs. Drill cases live in ~/.rmagent/drill/. Payloads stay under the WinRM ~8191-char budget.
The attack this simulates (the Ada story, class 03 "Walk")
The drill stages the exact kind of movement rmagent is built to catch — identity-led, living-off-the-land, no new malware. Each artifact maps to a specific rmagent signal:
| Staged artifact | Windows event | rmagent signal it should fire |
|---|---|---|
| 2 failed Administrator logons | 4625 | attest.admin_failed_60s, sketch.admin_failed |
New local admin RMAgentDrill_Test |
4720 + 4732 | explain.identity_changes, sketch.new_local_admins |
New SYSTEM scheduled task RMAgentDrill_Task |
4698 | explain.task_events, sketch.new_tasks |
New LocalSystem service RMAgentDrillSvc |
7045 | explain.service_events, sketch.new_services |
| PowerShell spawns | 4688 (if Process Creation audited) | explain.proc_spawns |
| SYSTEM outbound connection to 1.1.1.1:80 | Sysmon EID3 | |
| Run key RMAgentDrill_RunKey | Registry state | attackmap (T1547.001) |
| IFEO debugger RMAgentDrill.exe | Registry state | attackmap (T1546.010) |
Prerequisites
Jump host: macOS, Linux, or Windows. pywinrm connects to the Windows targets from any. On a non-macOS jump host, the
secretsscrt master password is provided viaSCRT_PASSenv var or a~/.scrt_passfile (macOS uses Keychain automatically). Thewinrm_pool.ps1RunspacePool path needs PowerShell 7.
- The
rmagent-windowsskill works (census passes on WS1/WS2). See its README. - Telegram alerting configured — token
telegram-bot-tokenand chattelegram-chat-idin thesecretsscrt store (already present), OR envRMAgent_TELEGRAM_TOKEN/RMAgent_TELEGRAM_CHAT. - Windows credentials — env
RMAgent_<ID>_PASS, OR the scrt keyswindows-server1-password/windows-server2-password(auto-loaded if env is missing). - Audit subcategories on (so Windows writes the events to its diary). On each box, run:
Without these, three of the six drill signals cannot be detected by any pull-based tool — Windows writes no event. On this estate they are already enabled.auditpol /set /subcategory:"Process Creation" /success:enable # 4688 process spawns auditpol /set /subcategory:"Other Object Access Events" /success:enable # 4698 scheduled tasks auditpol /set /subcategory:"Security Group Management" /success:enable # 4732 group-add (new local admin) — NOT "Account Management" (that's 4720) - Sysmon NetworkConnect on (for the transient outbound-connection signal): Sysmon is already installed on WS1/WS2; enable its network-connection ring with a minimal config (
<NetworkConnect> to log all), applied viaSysmon64.exe -c <config>. Then thenetedgesquestion reads the Sysmon EID3 ring. Without it,system_outbound_connshows 0 — a point-in-timeedgessnapshot misses sub-second connections. On this estate it is already enabled.
Run
export SKILL_DIR=~/.agents/skills/rmagent-redteam
cp ~/.agents/skills/rmagent-windows/assets/inventory.example.yaml ./estate.yaml
# Full loop: stage -> rmagent census+hunt -> score -> telegram -> clean
python3 "$SKILL_DIR/scripts/redteam.py" run --inventory ./estate.yaml --confirm
# Or step by step:
python3 "$SKILL_DIR/scripts/redteam.py" stage --inventory ./estate.yaml --confirm # stage artifacts
python3 "$SKILL_DIR/scripts/redteam.py" clean --inventory ./estate.yaml # remove them
--keep leaves artifacts staged after a run (clean later with clean).
What happens during a run
- Telegram: "drill started" message listing the artifacts it will stage.
- Stage
drill.ps1on each box. - Wait 8s for events to land in the logs.
- Run
rmagent-windowscensus + hunt (1h window) into a case folder. - Score detected vs staged — compares what rmagent recorded to the 8 expected signals.
- Telegram: a detection report —
Detected (N/8)with each signal, plus any missed, plus the case name. - Clean
clean.ps1on each box (unless--keep). - Telegram: "artifacts cleaned."
Output you owe the user
After a run, state plainly:
- What was staged (the 6 artifacts).
- What rmagent detected (N/6, with each signal and which rmagent field fired).
- What was missed and why (e.g., Process Auditing off → no 4688).
- The case path so the walk can be re-read.
- That artifacts were cleaned (or kept).
Laya-assisted drill gate and gap routing
Two recurring judgments can be made by a fast, typed decision model (Laya, via
the use-laya skill). Two matrices live in decisions/:
drill_gate.json— before staging a drill variant: reversible (noul) / scoped to the drill boxes only (noul) / gate (stage, show the operator the plan first, or refuse). It catches over-broad staging before a human sees it.gap_route.json— after a scored drill, over the detection scorecard: which of the LOTL techniques to drill next (or hand the choice to the LLM).
scripts/laya_decide.py drill_gate --state-file drill.json
scripts/laya_decide.py gap_route --state-file scorecard.json
Policy: confidence below 0.5 is flagged ESCALATE. Verdicts are advisory — every drill still needs explicit operator approval, artifacts stay reversible, and production boxes remain untouched.
Needle tier 0 — offline extraction and drift
Alongside the Laya matrices, every skill in this family can call needle — a 121M on-device model (Cactus Compute Needle, ~35 MB, no network, no keys, ~100 MB RAM) installed once on the jump host, never on the witnesses. It is the free tier of the judgment stack: needle extracts and compares offline, Laya judges, the LLM reasons.
Two scripts ship in scripts/:
needle_extract.py— typed field extraction from a witness fragment (count,source_ip, …). Rigid tokens (numbers, IPs) are reliable; semantic fields are not, so the wrapper validates what it can (:ip,:int) and blanks anything it cannot trust — never a guess.echo "<attest answer>" | scripts/needle_extract.py user count:int source_ip:ip --jsonneedle_drift.py— baseline-drift detection via needle's 3072-dim embeddings: record what an answer normally looks like, score today's answer against it (similar / drifted / changed, thresholds calibrated empirically). Separates shape and topic, not field values — value drift stays the job of the field rules and correlate joins.echo "<routine answer>" | scripts/needle_drift.py record <host>-<question> echo "<today's answer>" | scripts/needle_drift.py score <host>-<question>
The engine runs in-process on the jump host (the wrapper finds a Python
with cactus-needle — set NEEDLE_PYTHON if it lives elsewhere). No port
is opened, nothing is installed on any witness, and the baselines are
kilobyte holes under ~/.rmagent/needle-drift/, not a lake. On an
air-gapped estate this tier keeps working — and the judgment tier with it: Laya is offline too (pre-seed its HF-cache checkpoint on air-gapped hosts), so no matrix judgment waits for a connected session.
Non-negotiables
- Authorised estate only. WS1/WS2, or boxes the operator administers. Never a partner/NIBSS/production-critical box without explicit written consent.
--confirmrequired forstageandrun. No silent staging.- Reversible.
clean.ps1removes everyRMAgentDrill_*artifact. Idempotent. Run it after every drill unless you used--keep. - Benign payloads only. A network test to
1.1.1.1:80and an echo to a temp file. No exfiltration, no persistence, no destructive action. - Telegram is the alert channel, not a data channel. Send detection summaries, never credentials, Event Logs, or case contents.
- Don't compete with EDR. Defender/CrowdStrike may alert on the drill too — that's good. Coordinate timing if your SOC is staffed.
Limits
proc_spawns(4688) needs Process Creation auditing on the target. If it shows 0, checkauditpol /get /subcategory:"Process Creation".system_outbound_connneeds the Sysmon NetworkConnect ring enabled (<NetworkConnect>). The point-in-timeedgessnapshot misses a sub-second connection that already closed; the Sysmon ring persists it. If it shows 0, confirm Sysmon's NetworkConnect config is on.- The drill runs as Administrator over WinRM, so the staged "failed Administrator logon" events are real 4625s for the local Administrator account.
The two sides, separately
The drill is a purple-team loop by default, but the red side and the blue side can now be run independently:
# RED SIDE ONLY — stage the attack, leave it live, test ANYTHING against it
# (rmagent, CrowdStrike, Defender, a SIEM query you are writing — the drill
# does not care who does the detecting)
python3 redteam.py stage --inventory estate.yaml --confirm
# ...hours or days later...
# BLUE SIDE ONLY — score rmagent against whatever is STILL on the boxes.
# Read-only: stages nothing, cleans nothing, needs no --confirm.
python3 redteam.py detect --inventory estate.yaml
# Clean up whenever you are done
python3 redteam.py clean --inventory estate.yaml
detect is honest about what it scores: before running the hunt it verifies
(read-only, via verify.ps1) which RMAgentDrill_* artifacts are actually
present, and only counts those. A half-cleaned or never-staged artifact can
never be reported as a detection miss. If nothing is present it says so and
refuses to run a meaningless test.
This closes the gap the welded run loop had: you could always test another
detection system against staged artifacts, but you could not measure
rmagent's detection against a scenario that was staged earlier, or that you
arranged yourself. detect also enables delayed detection testing —
stage today, detect tomorrow, to answer "would we have caught this attack
while it sat on the box overnight?"
--keep leaves artifacts staged after a run (clean later with clean).
Enterprise scope & rules of engagement
- This is a DRILL, not an attack: all artifacts are prefixed
RMAgentDrill_, reversible withclean, and--confirmis mandatory. Only against boxes you administer, only inside your change window. - Sizing: staging 8 LOTL artifacts on 2 witnesses is minutes; detection
scoring is immediate;
--keep+ delayeddetectanswers "would we have caught this overnight?" — the question that actually matters. - Cadence: quarterly (align with the rmagent-at ring/watchdog drill so the capture plane is exercised under the same storm). Every drill ships a Telegram/report summary — detection gaps become tickets, not anecdotes.
- Blue/green contract: red stages with prefixes and cleans up; blue scores honestly (blind witnesses score as gaps, not passes). The drill's output is the input to sizing/retention fixes — not a scoreboard.
- No-Python estates: the drill stages artifacts via OS-native means (cmd/PowerShell one-liners) — the scoring half needs the engine on the jump host, which is your controlled runtime.