Mantis Pipeline Designer (/mantis-pipeline-adapter)
System Goal
Interactive Pipeline Design Consultant. Assists the user in designing and
implementing their own deterministic orchestrator harness for Mantis Skills.
Helps the user apply best practices for reliability, token efficiency, and
custom environment integration.
Command Definition
- Command:
/mantis-pipeline-adapter
- Description: Interactively guides the design and implementation of custom
deterministic orchestrator harnesses.
Input/Output Contract
- Reads:
workspace/.mantis_state.json (to track current loop pass).
workspace/.mantis_state.json fields active_snapshot, snapshot_history,
and vcs_info.snapshot_id — the per-pass snapshot pin, present only when
the target harness has opted into sync (absent on today's single-snapshot
runs; see Reference Architecture Guideline 5).
schema.json (as the canonical pipeline specification reference).
workspace/findings/*.json (as the State Store).
workspace/learnings.jsonl (to understand memory rotation).
- User's interactive configuration input.
- Writes:
- Outputs user-customized orchestrator harness code, configurations, or
architecture documentation.
- Preconditions:
- User initiates interactive design session.
- Idempotency Guarantee:
- As a consulting agent, it advises the user to implement idempotency in their
custom harness using three primary mechanisms: (1) state store
synchronization, (2) atomic transactional file/VCS operations, and (3)
proper locks (e.g. database/file level locks).
Instructions
Interactively guide the user in designing and building a deterministic pipeline
that wraps Mantis Skills.
Follow these guidelines during the consultation:
- Understand User Context: Ask about their target programming language,
agent framework (if any), execution environments (VMs, local containers,
physical hardware), and scale requirements.
- Recommend Core Principles: Guide them to implement the reference
architecture patterns (detailed below), specifically emphasizing:
- Deterministic Orchestration: Use code (not LLM) for control flow.
- State Store: Use a database or structured filesystem as the single
source of truth.
- Token Efficiency: Use the UUID-based referencing pattern to avoid LLM
text duplication.
- Custom Environment Integration: Use Custom MCP servers for isolated
testing (VMs) or hardware interaction.
- Ensure Schema Consistency: Advise the user to strictly adhere to the
inter-stage data contracts defined in schema.json when
building their harness.
- Adaptive Design: Help them draft the code/architecture tailored to their
specific stack, rather than imposing a rigid template.
- Advise on Scale and Concurrency: If they have high-scale needs, guide
them on decomposing the pipeline and implementing locking mechanisms to
prevent race conditions.
- Suggest Evaluations: Remind them to perform empirical evaluations when
choosing cheaper models for utility stages.
- Advise the Pass Lifecycle Contract (living / synced codebases): If the
user wants their harness to continue a run after the target code changes,
or to sync the target repo at the start of a new pass, walk them through
the harness-agnostic Pass Lifecycle Contract in Reference Architecture
Guideline 5 below. Emphasize that this support is opt-in: a harness that
does not implement the contract MUST leave
snapshot_pinned unset, which
preserves today's single-snapshot behavior byte-for-byte. When --sync is
requested, the harness PINs in the PIN step and passes
--snapshot_root/--snapshot_id normally; Block A (Locator Resolution) is
universal across all code-reading stages.
- Advise on Semantic Retrieval at Scale: If the user is targeting a large
codebase (e.g., thousands of source files, multi-pass campaigns, or multiple
teams contributing findings), walk them through the optional semantic
retrieval patterns in Reference Architecture Guidelines 6 and 7 below.
Emphasize that these are opt-in: they augment the pipeline via a
dedicated query skill or MCP tools, but never modify the existing skills'
own deterministic logic or fail-safe invariants.
- Advise on SAST Seeding: If the user wants to augment LLM-based discovery
with external SAST tool findings (CodeQL, Semgrep, etc.), walk them through
the optional SAST seeding pattern in Reference Architecture Guideline 8
below. Emphasize that this is opt-in: it ingests external findings as
candidates that must earn their verdict through unchanged downstream gates,
and it follows exactly the RAG pattern (provenance-tracked, snapshot-aware,
fallback on failure).
- Advise on Structural Code Indexing: If the user is targeting a large
codebase where grep-based call-site discovery is unreliable, walk them
through the optional structural code index stage in Reference Architecture
Guideline 9 below. Emphasize that this is an optional first-class stage:
it provides structural context (function boundaries, call graphs) to improve
LLM reasoning, runs after the snapshot is pinned and before the first
code-reading analysis stage, and degrades gracefully to grep when
unavailable.
- Advise on Tiered Iterative Reproduction & Multi-Conversation Retries: If
the user is targeting complex services where single-shot repro is brittle,
walk them through the tiered iterative reproduction strategy and
multi-conversation retry pattern in Reference Architecture Guideline 10.
Reference Architecture Guidelines
Use the following guidelines as your technical reference when advising the user.
Core Principles
- Deterministic Orchestration: Do not let the LLM decide the control flow
of the pipeline. Use a programmatic harness to call skills sequentially or in
parallel.
- State on Disk / Database: Use the filesystem
(
workspace/findings/*.json) or a database as the single source of truth.
Skills should read from and write to this store. For horizontal scaling,
recommend a centralized database.
- Deterministic Reporting: Treat findings as internal state. Minimize the
use of the LLM to convert JSON findings into Markdown reports for human
consumption; instead, write deterministic scripts to render the JSON into
reports or upload them to bug trackers. Only use an LLM for non-deterministic
subsets of this (like textual synthesis), such as by providing an executive
summary if necessary.
- Token Efficiency & Reusable Deterministic Tools: Structure LLM outputs to
return only the minimum necessary information (e.g., UUIDs, status codes).
Do not force the LLM to write one-off scripts (e.g., Python or bash) on the
fly for routine tasks like appending JSON fields or merging findings, as this
wastes reasoning tokens. Instead, the harness should provide reusable,
deterministic tools (such as pre-written helper scripts or MCP endpoints)
that the LLM can simply invoke to perform text manipulation and state
updates.
- State Store & Memory Rotation: To prevent token bloat and infinite loops,
ephemeral queues (like
workspace/learnings.jsonl) must be rotated. Upon
successful completion and verification of the Knowledge Base synthesis stage,
the orchestrator should ensure the archive directory exists (e.g.,
mkdir -p workspace/archive/learnings/) and move
workspace/learnings.jsonl to a numbered archive (e.g.,
workspace/archive/learnings/learnings_pass_${N}_${X}.jsonl where ${N} is
the loop pass and ${X} is a sub-index). If the synthesis fails, the active
queue must be left intact to prevent data loss.
Architectural Overview
graph TD
Harness[Programmatic Harness / Orchestrator] <--> DB[(State Store: Disk/DB)]
subgraph Stages [Decomposed Stages]
KB[KB Architect]
TM[Threat Modeler]
P[Plan]
R[Researcher]
D[Deduplicator]
V[Validator/Review]
C[Critic]
Rep[Reproducer]
Ch[Chainer]
Pat[Patcher]
Cal[Calibrator]
Ref[Reflector]
end
Harness --> KB
Harness --> TM
Harness --> P
Harness --> R
Harness --> D
Harness --> V
Harness --> C
Harness --> Rep
Harness --> Ch
Harness --> Pat
Pat -.->|Re-attack Bypass Loop| Rep
Harness --> Cal
Harness --> Ref
subgraph LLM Pool [Tailored LLMs]
ModelA[Frontier Model: Deep Reasoning]
ModelB[Flash/Lite Model: Fast & Cheap]
ModelC[Alternative Provider: Diversified Logic]
end
KB -.-> ModelA
TM -.-> ModelB
P -.-> ModelB
R -.-> ModelA
R -.-> ModelC
D -.-> ModelB
V -.-> ModelB
C -.-> ModelA
Rep -.-> ModelA
Ch -.-> ModelA
Pat -.-> ModelA
Cal -.-> ModelB
Ref -.-> ModelB
1. UUID-Based Referencing Pattern
To prevent the LLM from repeating large blocks of text (which increases latency,
cost, and the risk of mangling data), use UUIDs as the primary key for all
findings.
A. Researcher Stage
- Action: Sweeps the codebase and identifies potential vulnerabilities.
- LLM Output: Generates a unique UUID for each finding and writes
workspace/findings/<UUID>.json containing the full details (matching the
standard schema in Mantis Researcher).
B. Deduplication Stage (Optimized)
Instead of asking the LLM to read all findings, merge them in context, and write
them back, use the following pattern:
Harness Action: Reads all workspace/findings/*.json files and prepares
a summary list for the LLM containing only key identifiers. To align with the
standard schema, map the code_paths array (which uses "file:line" format)
to a simplified summary for the LLM:
[ { "id": "UUID", "file": "path", "line": 12, "snippet": "..." } ].
LLM Action: Analyzes the summary and outputs a mapping of duplicates:
{
"primary_uuid_1": ["duplicate_uuid_a", "duplicate_uuid_b"],
"primary_uuid_2": []
}
Harness Action (Deterministic):
- Reads the content of the affected files.
- Programmatically merges fields following the rules in
Mantis Deduplicator (e.g., union of
code_paths, taking highest severity, concatenating history).
- Updates
workspace/findings/primary_uuid_1.json on disk.
- Ensures the trash directory exists (e.g.,
mkdir -p workspace/findings/.trash/).
- Moves
workspace/findings/duplicate_uuid_a.json and
workspace/findings/duplicate_uuid_b.json to the trash staging directory
(workspace/findings/.trash/).
C. Validation & Review Stages (Reviewer, Critic)
- Harness Action: For each finding
workspace/findings/<UUID>.json, pass
only the relevant code context and finding description to the LLM.
- LLM Action: Output only a structured verification result (e.g.,
{"valid": true, "reason": "..."}).
- Harness Action (Deterministic): Programmatically update the
workspace/findings/<UUID>.json file with the validation status and reason.
2. Adaptable Reproducers via Custom MCP
When validating findings, the agent may need to interact with diverse
environments (VMs, physical hardware). Use the Model Context Protocol (MCP)
to expose a clean, restricted API.
- Architecture:
[Reproducer Agent] <--- MCP ---> [Custom MCP Server] <--- API ---> [Target Env]
- Custom Environments:
- VMs: Implement tools like
reboot_vm(), execute_payload().
- Hardware/USB: Implement tools like
power_cycle_device() (via smart
plug), send_usb_packet().
- Integration Note: If the user's harness uses raw LLM APIs (e.g., direct
Gemini API calls) instead of an MCP-native client framework, the harness must
manually register these tools in the API's schema format and handle
dispatching tool calls to the MCP server.
3. Decomposition & Multi-Model Strategy
A. Pipeline Decomposition & Concurrency
The pipeline can be split into independent services. When scaling horizontally
(e.g., multiple workers running the Reproducer stage in parallel):
- Concurrency Control: Implement database or file locking to ensure two
workers do not attempt to process or update the same finding simultaneously.
- Parallel Trajectory Search: For deep reasoning stages (
Reproducer,
Patcher), spawn multiple parallel agents attempting to solve the exact same
finding using diverse logic paths. For the Reproducer stage, prune all other
trajectories as soon as one worker succeeds to save compute costs while
escaping LLM "give up" loops. For the Patcher stage, wait for all patches to
be generated and tested, then evaluate the successful ones to select the most
minimal, idiomatic, and correct fix.
B. Heterogeneous LLM Selection (Multi-Model)
Match task complexity with the appropriate model tier:
- Frontier Models: For deep reasoning (Research, Reproduce, Patch).
- Flash/Lite Models: For structured utility tasks (Dedupe, Calibrate).
- Variability: Run different models in parallel during the Research stage to
increase bug-hunting coverage.
C. Importance of Evaluation
Emphasize that using cheaper models for utility stages (like deduplication or
calibration) must be validated with empirical evaluations against a benchmark
dataset to ensure quality is not degraded.
4. The Planning Stage and workspace/plan.json
The planning stage plays a critical role in structuring the security campaign.
The strategist (/mantis-plan) generates workspace/plan.json to define
targeted investigations, context pointers, and specific questions for the
auditor. The researcher (/mantis-researcher) reads workspace/plan.json at
startup to guide its sweep. By decoupling strategy and execution via this
structured contract, the orchestrator can easily direct subagents, parallelize
sweeps, and maintain historical context across pipeline runs without repeating
work.
5. The Pass Lifecycle Contract (Living / Synced Codebases)
A custom orchestrator (a bespoke CLI, an ADK agent, an MCP-native pipeline, or
any deterministic harness) does not inherit the living-project lifecycle
that mantis-meta-agent implements. To support continue-after-edits and
opt-in boundary sync without producing silent wrong results (false
VERIFIED_SECURE, false failed_to_reproduce, dropped regressions), the
harness must implement the following harness-agnostic contract. This is the same
contract recorded in schema.json under Non-JSON Contracts;
the Block A–Block G and SNAPSHOT_ID references below name mechanisms each
Mantis stage already carries in its own SKILL.md.
Mantis runs under multiple harnesses (various CLIs, ADK, custom deterministic
pipelines), so the lifecycle must not live only in mantis-meta-agent. Any
harness is conformant iff, per pass, it:
- SYNCs first (Block C) — the very first action; never mid-pass.
- Detects
vcs_info + computes SNAPSHOT_ID (Block D steps 1-5) — only
after sync.
- PINs the immutable copy + writes the sentinel + appends
snapshot_history (Block D step 5, not RECORD).
- Records
vcs_info (incl. snapshot_id) + active_snapshot. Never
record an id or pin before syncing.
- Runs every stage with
--snapshot_root=<SNAPSHOT_ROOT> --snapshot_id=<SNAPSHOT_ID> --state_root=<workspace parent>.
- Archives & increments (existing Stage 15); retried findings keep their
original
discovery_commit.
A harness that does not implement the contract MUST leave snapshot_pinned
unset → today's behavior. When --sync is requested, the harness PINs in the
PIN step and passes --snapshot_root/ --snapshot_id normally; Block A
(Locator Resolution) is universal across all code-reading stages.
Advisory notes when helping a builder implement this contract
- Opt-in, default off. Sync/pinning is a feature the builder turns on. A
harness that never sets
snapshot_pinned behaves exactly like today (one live
snapshot per run). Downstream stages treat an absent
active_snapshot/discovery_commit as the conservative branch, so an
un-upgraded harness is always safe — just not living-project-aware. Do not
advise treating these absent fields as an error.
- Store snapshots OUTSIDE
workspace/. The pinned copy (SNAPSHOT_ROOT)
must live under <state_root>/.mantis_snapshots/pass_<N> (or a clean-VCS
worktree/archive), and its path must not contain the segment /workspace/
— otherwise mantis-patch's state-vs-code path guard misfires. Keep the last
2 snapshots and garbage-collect older ones with the matching teardown
(rm -rf for copies, git worktree remove/prune for worktrees).
- Non-destructive sync only. Sync is the first action of a pass,
never mid-pass, and must be skipped when the tree is dirty, ahead of
upstream, detached, or has no upstream. The harness must never run
git reset --hard, git checkout -- ., git clean, or hg update -C, or
any command that discards uncommitted/untracked/local-commit state — user
edits and in-progress work must survive every pass.
- Full-fidelity
SNAPSHOT_IDs, including dirty / no-VCS. Compute the id
over the whole pinned copy: clean git/hg → commit_hash; dirty git/hg →
commit_hash + ":" + content_hash; multi-vcs →
revision + ":" + content_hash; no-VCS / unknown copyable tree →
"content:" + content_hash. The embedded content hash is exactly what lets an
unchanged dirty or no-VCS tree MATCH across passes and still receive
verification + dedup — and what makes a repo sync that advances commits
under an unchanged manifest revision compare unequal. Never trust a bare
branch name or manifest revision string as an identity.
- Pass the three roots to EVERY stage. Include the findings-only stages
(report, calibrate, reflect): they do not read target code, but they still
read
active_snapshot for provenance/annotation. When the harness archives
and increments, retried findings must keep their original
discovery_commit.
Conformance scenarios
The scenarios below expose nearly every issue in the snapshot model. They are
reference checks, not features: the harness is responsible for preventing or
handling each one in its own environment. The table is a quick-reference; prose
detail follows for each scenario. The State column uses the 3-STATE RULE
(MODE-OFF / HALT / PINNED, branched on active_snapshot presence — see the
global backward-compat rule in schema.json and the advisory
notes above); SNAPSHOT_ID formats follow the ladder in the advisory notes
above (e.g. live:<ts> signals an unpinned/HALT pass).
Invariant legend (the labels below name safety properties enforced by the
blocks and the global backward-compat rule in schema.json):
| Label |
Property |
Enforced by |
| INV-1 |
No false VERIFIED_SECURE |
Block G + HALT ceiling |
| INV-2 |
No false failed_to_reproduce |
Block F + HALT ceiling |
| INV-3 |
No dropped regression |
Block B NOT_MATCHED + POSSIBLE REGRESSION |
| INV-4 |
Within-pass consistency |
Block A sentinel + single pinned snapshot |
| INV-5 |
No user data loss |
Block C non-destructive sync + Block A step 4 |
| INV-6 |
Fail-safe on missing data |
Global backward-compat rule |
Quick-reference table:
| # |
Scenario |
State |
Harness behavior |
Stage behavior |
Block / INV |
Key fields |
| 1 |
Colocated state |
PINNED |
HALT-and-yield (safe default), or relocate state_root outside CODE_ROOT when explicitly authorized (e.g. --auto_relocate_state); SNAPSHOT_ROOT path must not contain /workspace/ |
mantis-patch state-vs-code guard misfires; Block A step 3 confuses SNAPSHOT- vs STATE-relative paths |
A:3, D:3; INV-5 |
active_snapshot.root, snapshot_root, state_root |
| 2 |
Stale active_snapshot (active_snapshot.pass != state.pass_number) |
PINNED → STOP or HALT-degrade |
Block D step 0: handles same-pass re-entry only; if dir missing → STOP, yield to user |
Block A step 2 sentinel may still MATCH (dir retained); CURRENT-PASS CHECK (active_snapshot.pass == state.pass_number) required: mismatch → STOP or HALT-degrade (Block B NOT_MATCHED, no authoritative verdicts) |
A:2, D:0, B; INV-1, INV-3, INV-4, INV-6 |
active_snapshot.{root, snapshot_id, snapshot_pinned, pass}, state.pass_number, discovery_commit |
| 3 |
Pin failure |
HALT |
Block D step 2/4: skip copy on ENOSPC/error → step 5b; still write active_snapshot + pass roots |
Authoritative verdicts forbidden; Block B always NOT_MATCHED; reproduce not_attempted; patch VERIFICATION_INCOMPLETE |
D:2, D:4, D:5b; INV-1, INV-2, INV-6 |
active_snapshot.{snapshot_id, snapshot_pinned} |
| 4 |
Patched shadows |
PINNED (pass); --snapshot_pinned=false arg |
Pass --target_root=<PATCHED_SHADOW_ROOT> + --snapshot_pinned=false to reattack sub-agent |
Block A step 1a: CODE_ROOT=--target_root (authoritative); step 2 sentinel SKIPPED (sentinel-EXEMPT) |
A:1a, A:2; INV-4 |
target_root, snapshot_pinned (arg), snapshot_root, discovery_commit |
| 5 |
Different-snapshot duplicate candidates |
PINNED |
No special action — both passes pinned correctly; dedupe handles it |
Block B pairwise: discovery_commit differs → NOT_MATCHED → keep ACTIVE + possible_duplicate_of; POSSIBLE REGRESSION if archived was RESOLVED |
B; INV-3, INV-6 |
discovery_commit, possible_duplicate_of, status, patch_status |
| 6 |
Absent sink evidence |
Any |
No special action — Block F is a stage-level mechanical gate |
Block F: evidence absent (build error, exit 127, sink unreached) → not_attempted (retry-eligible), NEVER failed_to_reproduce; HALT ceiling additionally forces not_attempted |
F; INV-2, INV-6 |
repro_status, reattack_status, repro_hints |
Per-scenario detail:
1. Colocated state (state_root nested inside CODE_ROOT / snapshot root)
— The pinned SNAPSHOT_ROOT must live under
<state_root>/.mantis_snapshots/pass_<N> (or a clean-VCS worktree/archive), and
its path must not contain the segment /workspace/ — otherwise
mantis-patch's state-vs-code path guard misfires (state files appear to be
"under CODE_ROOT"). If state_root itself is inside CODE_ROOT, the harness
must HALT-and-yield (safe default) or, when explicitly authorized (e.g.
--auto_relocate_state), relocate it outside the snapshot before pinning. Block
A step 3 distinguishes SNAPSHOT-RELATIVE path fields (read under CODE_ROOT)
from STATE-RELATIVE fields (read under state_root/workspace, never prefixed
with CODE_ROOT); colocation breaks this separation.
2. Stale active_snapshot (active_snapshot.pass != state.pass_number —
active_snapshot was preserved across the Stage 15 pass increment) — Block D
step 0 (crash-resume) handles only the SAME-pass re-entry case
(active_snapshot.pass == N → reuse). It does NOT catch a stale snapshot
carried across the Stage 15 pass increment, because Stage 15 deliberately
preserves active_snapshot while bumping pass_number (see Stage 15). Two
sub-cases:
(a) The prior snapshot dir is now MISSING: Block D step 0 STOPs and yields to
the user (never re-pin to a possibly-drifted live tree). (b) The prior snapshot
dir still EXISTS (default keep-2 retention) and its sentinel matches the
preserved active_snapshot.snapshot_id: Block A step 2 sentinel check SUCCEEDS
(it only compares the sentinel file to SNAPSHOT_ID, not to the current pass).
Block B's pairwise discovery_commit check would MATCH a carried-forward
finding against a new finding stamped with the same stale SNAPSHOT_ID,
silently dropping it as DUPLICATE — a false authoritative verdict.
To prevent (b), the HARNESS MUST guarantee that
active_snapshot.pass == state.pass_number before any consumer stage reads it.
The reference harness (mantis-meta-agent) satisfies this by re-pinning every
pass (Block D step 0 sees active_snapshot.pass != N → re-pins → refreshes
active_snapshot.pass before any stage runs), so sub-case (b) never fires
there. A custom harness that preserves active_snapshot across the Stage 15
pass increment WITHOUT re-pinning MUST either (a) re-pin every pass (the
reference behavior), or (b) inject an equivalent pre-stage gate that refreshes
active_snapshot.pass or clears active_snapshot entirely before invoking
stages. Stages CANNOT self-detect this staleness via Block B (which is
snapshot_id-only, not pass-aware): a carried-forward finding and a new
finding stamped with the same stale SNAPSHOT_ID will MATCH in Block B despite
the snapshot being stale. The active_snapshot.pass field is defined in
schema.json #/$defs/state/active_snapshot/pass for exactly this check. The
harness's Block D step 0 reuse check is NOT a substitute: it only fires on
same-pass re-entry. (Stages that read active_snapshot MAY additionally
self-check defensively — see each stage's Step 0 sentinel check — but the
binding guarantee is on the harness.)
3. Pin failure (snapshot copy fails — disk full, permissions, too-large
tree) — Block D step 2 (free-space precheck): compare du -s of the live tree
to df free space at state_root; if it won't fit → skip copy → step 5b. Block
D step 4 (failure-tolerant verify): check copy exit status + sanity check (file
count/size within ~90%); on failure → step 5b (unpinned/HALT). Step 5b:
SNAPSHOT_ROOT=<live root>, snapshot_pinned=false,
SNAPSHOT_ID="live:"+ISO8601. The harness still writes active_snapshot and
still passes --snapshot_root/--snapshot_id to stages so they see the HALT
signal. Every stage then degrades conservatively: authoritative verdicts
forbidden (VERIFIED_SECURE, failed_to_reproduce, DUPLICATE,
FALSE_POSITIVE, NON_VIABLE); Block B always returns NOT_MATCHED; reproduce
records not_attempted; patch's best attainable is VERIFICATION_INCOMPLETE.
4. Patched shadows (--target_root pointing at a pre-mutated tree;
sentinel-exempt path 1a in Block A) — mantis-patch passes
--target_root=<PATCHED_SHADOW_ROOT> and --snapshot_pinned=false to the
reproduce sub-agent for re-attack verification. Block A step 1a:
CODE_ROOT = --target_root (authoritative override, overrides --snapshot_root
and state fallback). Block A step 2: sentinel check SKIPPED (a --target_root
tree is deliberately mutated and is sentinel-EXEMPT). The
--snapshot_pinned=false argument is the sentinel-exemption, NOT a HALT signal
— detect HALT by reading STATE (active_snapshot.snapshot_id starts with
live:, equivalently active_snapshot.snapshot_pinned is false in state),
never from the argument passed on this invocation. The finding's
discovery_commit is unaffected — it retains the pass-level SNAPSHOT_ID from
when it was discovered; only the --snapshot_pinned=false argument is local to
the reattack invocation.
5. Different-snapshot duplicate candidates (cross-pass dedupe where
discovery_commit differs — the pairwise Block B NOT_MATCHED path) — Both
passes pinned correctly; the findings simply come from different snapshots.
mantis-dedupe Block B pairwise check compares the CURRENT finding's
discovery_commit against the ARCHIVED finding's discovery_commit (NOT
against the global SNAPSHOT_ID). If they differ → NOT_MATCHED. NOT_MATCHED
keeps the current finding ACTIVE and sets possible_duplicate_of (a soft,
non-terminal hint — the finding is NOT filtered or trashed). If the archived
finding was RESOLVED (patch_status in {VERIFIED_SECURE,
MITIGATION_PROPOSED} OR status==FALSE_POSITIVE OR
production_viability==NON_VIABLE) AND the pair is NOT_MATCHED → POSSIBLE
REGRESSION: keep ACTIVE, add a history note, never filter (a reverted fix
re-discovered on new code must never be trashed).
6. Absent sink evidence (Block F — PoC compiles but produces no reached-sink
evidence; not_attempted vs failed_to_reproduce) — mantis-reproduce Block
F: if EVIDENCE is ABSENT (any compiler/build nonzero exit, exit 127
command-not-found, exit 2 "No such file", or the sink was never reached) →
repro_status = not_attempted (retry-eligible), STOP. NEVER
failed_to_reproduce. In --reattack mode: leave reattack_status UNSET with
a history note "setup_failed" — NEVER failed_to_bypass. failed_to_reproduce
is reserved for when the harness PROVABLY reached the vulnerable entrypoint —
i.e. reached-sink evidence, not setup evidence — but the bug did not fire.
Reached-sink evidence must originate INSIDE the invoked path or from
target-produced tracing/backtraces: (a) a PoC script/source harness writes
MANTIS_REACHED_ENTRYPOINT to a sidecar file at the point just before the sink
call, within its own execution flow (the marker write is part of the invoked
path, not a pre-launch step); OR (b) for binary/firmware/raw-payload targets,
the captured crash backtrace or sanitizer trace (ASan/UBSan/MSan/TSan)
explicitly names the target sink function (target-produced tracing). A marker
written by an external wrapper BEFORE invoking the target is SETUP EVIDENCE ONLY
(proves "launch attempted," not "sink reached") and does NOT by itself justify
failed_to_reproduce — treat it as EVIDENCE ABSENT for the decision gate.
Evidence is recorded in repro_hints. In HALT mode, the HALT ceiling
additionally forces not_attempted (no failed_to_reproduce), since a negative
result on an unpinned tree cannot be trusted as authoritative.
6. Semantic Retrieval (RAG) for Large Codebases
For small repositories, the planner can manually scan workspace/kb/index.md
and the researcher can grep for call-sites. At scale (thousands of files, deep
directory trees, multi-pass campaigns), these approaches miss relevant context
and waste tokens reading irrelevant files. A semantic retrieval layer lets the
planner and researcher query for relevant KB entries and code locations without
reading everything.
Two implementations are supported, sharing the same data contract:
- Option A (Default — Skill-Based): A dedicated skill that runs a
BM25/TF-IDF helper script over
chunks.jsonl. Zero external dependencies —
works air-gapped, no vector embeddings or vector store required. Optional
vector embedding support if available.
- Option B (Maximum Scale — MCP-Based): The harness owns a persistent vector
index using vector embeddings, serving persistent
semantic_search_kb /
semantic_search_code MCP tools. Better for very large codebases where
per-invocation BM25 is too slow.
Both are opt-in. The existing skills are not modified; the planner and
researcher receive runtime instructions to use whichever retrieval mechanism is
available, falling back to today's manual behavior if neither is present.
Retrieval results are coverage HINTs only — they decide ordering and
prioritization, never the membership of the audit set. A miss must never cause a
file, call-site, or investigation to be skipped or dropped.
A. Shared Data Contract: chunks.jsonl
After Stage 2 (/mantis-architecture) completes, chunks are extracted into
workspace/kb/chunks.jsonl (one JSON object per line). The harness can do this
post-hoc by reading workspace/kb/*.md, or the architecture skill can be
instructed to write it during synthesis as a text-only side effect. Two chunk
types are produced:
KB chunks from the existing workspace/kb/*.md files:
{"id": "auth_module:0", "source_file": "workspace/kb/entities/auth_module.md", "entity_type": "entity", "chunk_text": "The auth module handles..."}
Code chunks from CODE_ROOT (the pinned snapshot). Each chunk includes
the file path and line range so the researcher can request specific files
from the snapshot:
{"id": "src/parser.c:0", "source_file": "src/parser.c", "start_line": 1, "end_line": 80, "chunk_text": "int parse_input(..."}
The first line of chunks.jsonl is a provenance header recording the
SNAPSHOT_ID the chunks were built against:
{"_provenance": true, "snapshot_id": "abc123", "kb_snapshot_id": "abc123"}
Before serving queries, check snapshot_id in the provenance header against the
current SNAPSHOT_ID; rebuild if they differ. In MODE-OFF (no
active_snapshot), kb_snapshot_id is never stamped — skip the index entirely
and let skills fall back to manual scanning. Never build code chunks from the
live tree — they must reflect the pinned copy the skills are reading.
B. Option A: Skill-Based Retrieval (Default — No Infrastructure)
A dedicated skill reads chunks.jsonl and writes+runs a helper script (e.g.
workspace/helpers/search_chunks.py) that performs BM25/TF-IDF similarity
search. The script is generated by the agent at runtime — no code is shipped
with the skill (same pattern as mantis-dedupe's merge_findings.py). This
requires zero external dependencies — no embedding model, no vector store, no
MCP server. It works in air-gapped and VPC-SC environments.
A complete reference blueprint for this skill is available at
references/mantis-kb-query.md. Builders can
adapt it to their environment. The blueprint includes Block A (Locator
Resolution), chunk provenance checking, the versioned helper script contract
(MANTIS_HELPER_VERSION = 1), and the JSON output schema.
- Invocation: The planner or researcher spawns the skill as a sub-agent with
a query string. The skill writes the helper if not already present, runs it,
and returns top-K matching chunks as JSON.
- Optional embeddings: If vector embeddings are available, the agent can be
instructed to use cosine similarity instead of BM25. This is a runtime
configuration toggle, not a different skill.
- Snapshot safety: The skill reads
active_snapshot from state via Block A
(same as every other skill) and checks chunk provenance before serving.
C. Option B: MCP-Based Retrieval (For Maximum Scale)
For very large codebases where per-invocation BM25 is too slow, the harness can
own a persistent vector index using vector embeddings, serving two MCP tools
(following the same pattern as Guideline 2's Custom MCP for VMs/hardware):
semantic_search_kb(query: string) → [{id, source_file, entity_type, chunk_text, score}]
— Searches KB chunks. Returns relevant entity/vulnerability markdown context.
semantic_search_code(query: string) → [{file, start_line, end_line, snippet, score}]
— Searches code chunks from the pinned snapshot. Returns relevant code
locations.
The harness manages the vector index lifecycle: build from chunks.jsonl (or
directly from CODE_ROOT), rebuild when SNAPSHOT_ID changes, and handle
freshness checks. In HALT mode, serve with a STALE flag or refuse. In
MODE-OFF, skip entirely.
D. Per-Skill Augmentation Guidance
When a retrieval mechanism (skill or MCP) is available, instruct the following
skills to use it. These are runtime instructions passed by the harness or
meta-agent when invoking the skill — the skill files themselves are not
modified:
mantis-architecture: No changes needed. The harness chunks the existing
workspace/kb/*.md files after the architect completes Stage 2. If the
builder prefers, they may instruct the architect to also write
workspace/kb/chunks.jsonl during synthesis (Step 3) as a text-only side
effect — but this is optional, since the harness can extract chunks post-hoc.
mantis-plan: If a retrieval mechanism is available, instruct the planner
to use it to discover kb_references for each investigation instead of only
manually scanning workspace/kb/index.md. For each investigation, query with
the investigation title and target file names, then add the top-K matching KB
entity/vulnerability files to the kb_references array. Manual scanning of
index.md remains the fallback when no mechanism is available.
mantis-researcher: If a retrieval mechanism is available, instruct Wave 1
sub-agents to use it to PRIORITIZE relevant call-sites and cross-module data
flows into sinks (e.g., "where does untrusted input reach memcpy in the
parser module"). Semantic search SUPPLEMENTS grep as a ranking HINT ONLY — it
decides ORDER, never MEMBERSHIP
…(truncated)
1---
2name: mantis-pipeline-adapter
3description: >-
4 Interactively guides the design and implementation of custom deterministic orchestrator harnesses.
5 Use when a user wants to build their own pipeline to wrap and run Mantis skills reliably.
6 Don't use for executing the default pipeline directly.
7metadata:
8 source: https://skillmd.com/skills/google/mantis-pipeline-adapter
9---
10
11# Mantis Pipeline Designer (/mantis-pipeline-adapter)
12
13## System Goal
14
15Interactive Pipeline Design Consultant. Assists the user in designing and
16implementing their own deterministic orchestrator harness for Mantis Skills.
17Helps the user apply best practices for reliability, token efficiency, and
18custom environment integration.
19
20## Command Definition
21
22- **Command:** `/mantis-pipeline-adapter`
23- **Description:** Interactively guides the design and implementation of custom
24 deterministic orchestrator harnesses.
25
26## Input/Output Contract
27
28- **Reads**:
29 - `workspace/.mantis_state.json` (to track current loop pass).
30 - `workspace/.mantis_state.json` fields `active_snapshot`, `snapshot_history`,
31 and `vcs_info.snapshot_id` — the per-pass snapshot pin, present only when
32 the target harness has opted into sync (absent on today's single-snapshot
33 runs; see Reference Architecture Guideline 5).
34 - `schema.json` (as the canonical pipeline specification reference).
35 - `workspace/findings/*.json` (as the State Store).
36 - `workspace/learnings.jsonl` (to understand memory rotation).
37 - User's interactive configuration input.
38- **Writes**:
39 - Outputs user-customized orchestrator harness code, configurations, or
40 architecture documentation.
41- **Preconditions**:
42 - User initiates interactive design session.
43- **Idempotency Guarantee**:
44 - As a consulting agent, it advises the user to implement idempotency in their
45 custom harness using three primary mechanisms: (1) state store
46 synchronization, (2) atomic transactional file/VCS operations, and (3)
47 proper locks (e.g. database/file level locks).
48
49## Instructions
50
51Interactively guide the user in designing and building a deterministic pipeline
52that wraps Mantis Skills.
53
54Follow these guidelines during the consultation:
55
5601. **Understand User Context:** Ask about their target programming language,
57 agent framework (if any), execution environments (VMs, local containers,
58 physical hardware), and scale requirements.
5902. **Recommend Core Principles:** Guide them to implement the reference
60 architecture patterns (detailed below), specifically emphasizing:
61 - **Deterministic Orchestration**: Use code (not LLM) for control flow.
62 - **State Store**: Use a database or structured filesystem as the single
63 source of truth.
64 - **Token Efficiency**: Use the UUID-based referencing pattern to avoid LLM
65 text duplication.
66 - **Custom Environment Integration**: Use Custom MCP servers for isolated
67 testing (VMs) or hardware interaction.
6803. **Ensure Schema Consistency**: Advise the user to strictly adhere to the
69 inter-stage data contracts defined in [schema.json](../schema.json) when
70 building their harness.
7104. **Adaptive Design**: Help them draft the code/architecture tailored to their
72 specific stack, rather than imposing a rigid template.
7305. **Advise on Scale and Concurrency**: If they have high-scale needs, guide
74 them on decomposing the pipeline and implementing locking mechanisms to
75 prevent race conditions.
7606. **Suggest Evaluations:** Remind them to perform empirical evaluations when
77 choosing cheaper models for utility stages.
7807. **Advise the Pass Lifecycle Contract (living / synced codebases):** If the
79 user wants their harness to *continue a run after the target code changes*,
80 or to *sync the target repo at the start of a new pass*, walk them through
81 the harness-agnostic Pass Lifecycle Contract in Reference Architecture
82 Guideline 5 below. Emphasize that this support is **opt-in**: a harness that
83 does not implement the contract MUST leave `snapshot_pinned` unset, which
84 preserves today's single-snapshot behavior byte-for-byte. When `--sync` is
85 requested, the harness PINs in the PIN step and passes
86 `--snapshot_root`/`--snapshot_id` normally; Block A (Locator Resolution) is
87 universal across all code-reading stages.
8808. **Advise on Semantic Retrieval at Scale:** If the user is targeting a large
89 codebase (e.g., thousands of source files, multi-pass campaigns, or multiple
90 teams contributing findings), walk them through the optional semantic
91 retrieval patterns in Reference Architecture Guidelines 6 and 7 below.
92 Emphasize that these are **opt-in**: they augment the pipeline via a
93 dedicated query skill or MCP tools, but never modify the existing skills'
94 own deterministic logic or fail-safe invariants.
9509. **Advise on SAST Seeding:** If the user wants to augment LLM-based discovery
96 with external SAST tool findings (CodeQL, Semgrep, etc.), walk them through
97 the optional SAST seeding pattern in Reference Architecture Guideline 8
98 below. Emphasize that this is **opt-in**: it ingests external findings as
99 candidates that must earn their verdict through unchanged downstream gates,
100 and it follows exactly the RAG pattern (provenance-tracked, snapshot-aware,
101 fallback on failure).
10210. **Advise on Structural Code Indexing:** If the user is targeting a large
103 codebase where grep-based call-site discovery is unreliable, walk them
104 through the optional structural code index stage in Reference Architecture
105 Guideline 9 below. Emphasize that this is an **optional first-class stage**:
106 it provides structural context (function boundaries, call graphs) to improve
107 LLM reasoning, runs after the snapshot is pinned and before the first
108 code-reading analysis stage, and degrades gracefully to grep when
109 unavailable.
11011. **Advise on Tiered Iterative Reproduction & Multi-Conversation Retries:** If
111 the user is targeting complex services where single-shot repro is brittle,
112 walk them through the tiered iterative reproduction strategy and
113 multi-conversation retry pattern in Reference Architecture Guideline 10.
114
115## Reference Architecture Guidelines
116
117Use the following guidelines as your technical reference when advising the user.
118
119### Core Principles
120
1211. **Deterministic Orchestration:** Do not let the LLM decide the control flow
122 of the pipeline. Use a programmatic harness to call skills sequentially or in
123 parallel.
1242. **State on Disk / Database:** Use the filesystem
125 (`workspace/findings/*.json`) or a database as the single source of truth.
126 Skills should read from and write to this store. For horizontal scaling,
127 recommend a centralized database.
1283. **Deterministic Reporting:** Treat findings as internal state. Minimize the
129 use of the LLM to convert JSON findings into Markdown reports for human
130 consumption; instead, write deterministic scripts to render the JSON into
131 reports or upload them to bug trackers. Only use an LLM for non-deterministic
132 subsets of this (like textual synthesis), such as by providing an executive
133 summary if necessary.
1344. **Token Efficiency & Reusable Deterministic Tools:** Structure LLM outputs to
135 return only the *minimum necessary information* (e.g., UUIDs, status codes).
136 Do not force the LLM to write one-off scripts (e.g., Python or bash) on the
137 fly for routine tasks like appending JSON fields or merging findings, as this
138 wastes reasoning tokens. Instead, the harness should provide reusable,
139 deterministic tools (such as pre-written helper scripts or MCP endpoints)
140 that the LLM can simply invoke to perform text manipulation and state
141 updates.
1425. **State Store & Memory Rotation:** To prevent token bloat and infinite loops,
143 ephemeral queues (like `workspace/learnings.jsonl`) must be rotated. Upon
144 successful completion and verification of the Knowledge Base synthesis stage,
145 the orchestrator should ensure the archive directory exists (e.g.,
146 `mkdir -p workspace/archive/learnings/`) and **move**
147 `workspace/learnings.jsonl` to a numbered archive (e.g.,
148 `workspace/archive/learnings/learnings_pass_${N}_${X}.jsonl` where `${N}` is
149 the loop pass and `${X}` is a sub-index). If the synthesis fails, the active
150 queue must be left intact to prevent data loss.
151
152### Architectural Overview
153
154```mermaid
155graph TD
156 Harness[Programmatic Harness / Orchestrator] <--> DB[(State Store: Disk/DB)]
157
158 subgraph Stages [Decomposed Stages]
159 KB[KB Architect]
160 TM[Threat Modeler]
161 P[Plan]
162 R[Researcher]
163 D[Deduplicator]
164 V[Validator/Review]
165 C[Critic]
166 Rep[Reproducer]
167 Ch[Chainer]
168 Pat[Patcher]
169 Cal[Calibrator]
170 Ref[Reflector]
171 end
172
173 Harness --> KB
174 Harness --> TM
175 Harness --> P
176 Harness --> R
177 Harness --> D
178 Harness --> V
179 Harness --> C
180 Harness --> Rep
181 Harness --> Ch
182 Harness --> Pat
183 Pat -.->|Re-attack Bypass Loop| Rep
184 Harness --> Cal
185 Harness --> Ref
186
187 subgraph LLM Pool [Tailored LLMs]
188 ModelA[Frontier Model: Deep Reasoning]
189 ModelB[Flash/Lite Model: Fast & Cheap]
190 ModelC[Alternative Provider: Diversified Logic]
191 end
192
193 KB -.-> ModelA
194 TM -.-> ModelB
195 P -.-> ModelB
196 R -.-> ModelA
197 R -.-> ModelC
198 D -.-> ModelB
199 V -.-> ModelB
200 C -.-> ModelA
201 Rep -.-> ModelA
202 Ch -.-> ModelA
203 Pat -.-> ModelA
204 Cal -.-> ModelB
205 Ref -.-> ModelB
206```
207
208### 1. UUID-Based Referencing Pattern
209
210To prevent the LLM from repeating large blocks of text (which increases latency,
211cost, and the risk of mangling data), use UUIDs as the primary key for all
212findings.
213
214#### A. Researcher Stage
215
216- **Action:** Sweeps the codebase and identifies potential vulnerabilities.
217- **LLM Output:** Generates a unique UUID for each finding and writes
218 `workspace/findings/<UUID>.json` containing the full details (matching the
219 standard schema in [Mantis Researcher](../mantis-researcher/SKILL.md)).
220
221#### B. Deduplication Stage (Optimized)
222
223Instead of asking the LLM to read all findings, merge them in context, and write
224them back, use the following pattern:
225
2261. **Harness Action:** Reads all `workspace/findings/*.json` files and prepares
227 a summary list for the LLM containing only key identifiers. To align with the
228 standard schema, map the `code_paths` array (which uses `"file:line"` format)
229 to a simplified summary for the LLM:
230 `[ { "id": "UUID", "file": "path", "line": 12, "snippet": "..." } ]`.
231
2322. **LLM Action:** Analyzes the summary and outputs a mapping of duplicates:
233
234 ```json
235 {
236 "primary_uuid_1": ["duplicate_uuid_a", "duplicate_uuid_b"],
237 "primary_uuid_2": []
238 }
239 ```
240
2413. **Harness Action (Deterministic):**
242
243 - Reads the content of the affected files.
244 - Programmatically merges fields following the rules in
245 [Mantis Deduplicator](../mantis-dedupe/SKILL.md) (e.g., union of
246 `code_paths`, taking highest severity, concatenating history).
247 - Updates `workspace/findings/primary_uuid_1.json` on disk.
248 - Ensures the trash directory exists (e.g.,
249 `mkdir -p workspace/findings/.trash/`).
250 - Moves `workspace/findings/duplicate_uuid_a.json` and
251 `workspace/findings/duplicate_uuid_b.json` to the trash staging directory
252 (`workspace/findings/.trash/`).
253
254#### C. Validation & Review Stages (Reviewer, Critic)
255
256- **Harness Action:** For each finding `workspace/findings/<UUID>.json`, pass
257 only the relevant code context and finding description to the LLM.
258- **LLM Action:** Output *only* a structured verification result (e.g.,
259 `{"valid": true, "reason": "..."}`).
260- **Harness Action (Deterministic):** Programmatically update the
261 `workspace/findings/<UUID>.json` file with the validation status and reason.
262
263### 2. Adaptable Reproducers via Custom MCP
264
265When validating findings, the agent may need to interact with diverse
266environments (VMs, physical hardware). Use the **Model Context Protocol (MCP)**
267to expose a clean, restricted API.
268
269- **Architecture**:
270 `[Reproducer Agent] <--- MCP ---> [Custom MCP Server] <--- API ---> [Target Env]`
271- **Custom Environments**:
272 - *VMs*: Implement tools like `reboot_vm()`, `execute_payload()`.
273 - *Hardware/USB*: Implement tools like `power_cycle_device()` (via smart
274 plug), `send_usb_packet()`.
275- **Integration Note**: If the user's harness uses raw LLM APIs (e.g., direct
276 Gemini API calls) instead of an MCP-native client framework, the harness must
277 manually register these tools in the API's schema format and handle
278 dispatching tool calls to the MCP server.
279
280### 3. Decomposition & Multi-Model Strategy
281
282#### A. Pipeline Decomposition & Concurrency
283
284The pipeline can be split into independent services. When scaling horizontally
285(e.g., multiple workers running the `Reproducer` stage in parallel):
286
287- **Concurrency Control**: Implement database or file locking to ensure two
288 workers do not attempt to process or update the same finding simultaneously.
289- **Parallel Trajectory Search**: For deep reasoning stages (`Reproducer`,
290 `Patcher`), spawn multiple parallel agents attempting to solve the exact same
291 finding using diverse logic paths. For the `Reproducer` stage, prune all other
292 trajectories as soon as one worker succeeds to save compute costs while
293 escaping LLM "give up" loops. For the `Patcher` stage, wait for all patches to
294 be generated and tested, then evaluate the successful ones to select the most
295 minimal, idiomatic, and correct fix.
296
297#### B. Heterogeneous LLM Selection (Multi-Model)
298
299Match task complexity with the appropriate model tier:
300
301- *Frontier Models*: For deep reasoning (Research, Reproduce, Patch).
302- *Flash/Lite Models*: For structured utility tasks (Dedupe, Calibrate).
303- *Variability*: Run different models in parallel during the Research stage to
304 increase bug-hunting coverage.
305
306#### C. Importance of Evaluation
307
308Emphasize that using cheaper models for utility stages (like deduplication or
309calibration) must be validated with empirical evaluations against a benchmark
310dataset to ensure quality is not degraded.
311
312### 4. The Planning Stage and workspace/plan.json
313
314The planning stage plays a critical role in structuring the security campaign.
315The strategist (`/mantis-plan`) generates `workspace/plan.json` to define
316targeted investigations, context pointers, and specific questions for the
317auditor. The researcher (`/mantis-researcher`) reads `workspace/plan.json` at
318startup to guide its sweep. By decoupling strategy and execution via this
319structured contract, the orchestrator can easily direct subagents, parallelize
320sweeps, and maintain historical context across pipeline runs without repeating
321work.
322
323### 5. The Pass Lifecycle Contract (Living / Synced Codebases)
324
325A custom orchestrator (a bespoke CLI, an ADK agent, an MCP-native pipeline, or
326any deterministic harness) does **not** inherit the living-project lifecycle
327that `mantis-meta-agent` implements. To support *continue-after-edits* and
328*opt-in boundary sync* **without producing silent wrong results** (false
329`VERIFIED_SECURE`, false `failed_to_reproduce`, dropped regressions), the
330harness must implement the following harness-agnostic contract. This is the same
331contract recorded in [schema.json](../schema.json) under **Non-JSON Contracts**;
332the `Block A`–`Block G` and `SNAPSHOT_ID` references below name mechanisms each
333Mantis stage already carries in its own `SKILL.md`.
334
335Mantis runs under multiple harnesses (various CLIs, ADK, custom deterministic
336pipelines), so the lifecycle must not live only in `mantis-meta-agent`. Any
337harness is **conformant** iff, per pass, it:
338
3391. **SYNCs first** (Block C) — the very first action; never mid-pass.
3402. **Detects `vcs_info` + computes `SNAPSHOT_ID`** (Block D steps 1-5) — only
341 after sync.
3423. **PINs** the immutable copy + writes the sentinel + **appends**
343 `snapshot_history` (Block D step 5, not RECORD).
3444. **Records** `vcs_info` (incl. `snapshot_id`) + `active_snapshot`. Never
345 record an id or pin before syncing.
3465. **Runs every stage** with
347 `--snapshot_root=<SNAPSHOT_ROOT> --snapshot_id=<SNAPSHOT_ID> --state_root=<workspace parent>`.
3486. **Archives & increments** (existing Stage 15); retried findings keep their
349 **original** `discovery_commit`.
350
351A harness that does not implement the contract MUST leave `snapshot_pinned`
352unset → today's behavior. When `--sync` is requested, the harness PINs in the
353PIN step and passes `--snapshot_root`/ `--snapshot_id` normally; Block A
354(Locator Resolution) is universal across all code-reading stages.
355
356#### Advisory notes when helping a builder implement this contract
357
358- **Opt-in, default off.** Sync/pinning is a feature the builder turns on. A
359 harness that never sets `snapshot_pinned` behaves exactly like today (one live
360 snapshot per run). Downstream stages treat an absent
361 `active_snapshot`/`discovery_commit` as the conservative branch, so an
362 un-upgraded harness is always safe — just not living-project-aware. Do **not**
363 advise treating these absent fields as an error.
364- **Store snapshots OUTSIDE `workspace/`.** The pinned copy (`SNAPSHOT_ROOT`)
365 must live under `<state_root>/.mantis_snapshots/pass_<N>` (or a clean-VCS
366 worktree/archive), and its path **must not** contain the segment `/workspace/`
367 — otherwise `mantis-patch`'s state-vs-code path guard misfires. Keep the last
368 2 snapshots and garbage-collect older ones with the matching teardown
369 (`rm -rf` for copies, `git worktree remove/prune` for worktrees).
370- **Non-destructive sync only.** Sync is the **first** action of a pass,
371 **never** mid-pass, and must be **skipped** when the tree is dirty, ahead of
372 upstream, detached, or has no upstream. The harness must **never** run
373 `git reset --hard`, `git checkout -- .`, `git clean`, or `hg update -C`, or
374 any command that discards uncommitted/untracked/local-commit state — user
375 edits and in-progress work must survive every pass.
376- **Full-fidelity `SNAPSHOT_ID`s, including dirty / no-VCS.** Compute the id
377 over the **whole** pinned copy: clean git/hg → `commit_hash`; dirty git/hg →
378 `commit_hash + ":" + content_hash`; multi-vcs →
379 `revision + ":" + content_hash`; no-VCS / unknown copyable tree →
380 `"content:" + content_hash`. The embedded content hash is exactly what lets an
381 **unchanged dirty or no-VCS tree MATCH across passes** and still receive
382 verification + dedup — and what makes a `repo sync` that advances commits
383 under an unchanged manifest `revision` compare **unequal**. Never trust a bare
384 branch name or manifest revision string as an identity.
385- **Pass the three roots to EVERY stage.** Include the findings-only stages
386 (report, calibrate, reflect): they do not read target code, but they still
387 read `active_snapshot` for provenance/annotation. When the harness archives
388 and increments, retried findings must keep their **original**
389 `discovery_commit`.
390
391#### Conformance scenarios
392
393The scenarios below expose nearly every issue in the snapshot model. They are
394**reference checks**, not features: the harness is responsible for preventing or
395handling each one in its own environment. The table is a quick-reference; prose
396detail follows for each scenario. The **State** column uses the 3-STATE RULE
397(MODE-OFF / HALT / PINNED, branched on `active_snapshot` presence — see the
398global backward-compat rule in [schema.json](../schema.json) and the advisory
399notes above); `SNAPSHOT_ID` formats follow the ladder in the advisory notes
400above (e.g. `live:<ts>` signals an unpinned/HALT pass).
401
402**Invariant legend** (the labels below name safety properties enforced by the
403blocks and the global backward-compat rule in [schema.json](../schema.json)):
404
405| Label | Property | Enforced by |
406| ----- | ------------------------------ | --------------------------------------------- |
407| INV-1 | No false `VERIFIED_SECURE` | Block G + HALT ceiling |
408| INV-2 | No false `failed_to_reproduce` | Block F + HALT ceiling |
409| INV-3 | No dropped regression | Block B NOT_MATCHED + POSSIBLE REGRESSION |
410| INV-4 | Within-pass consistency | Block A sentinel + single pinned snapshot |
411| INV-5 | No user data loss | Block C non-destructive sync + Block A step 4 |
412| INV-6 | Fail-safe on missing data | Global backward-compat rule |
413
414**Quick-reference table:**
415
416| # | Scenario | State | Harness behavior | Stage behavior | Block / INV | Key fields |
417| --- | ------------------------------------------------------------------- | -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------- | ----------------------------------------------------------------------------------------------------- |
418| 1 | Colocated state | PINNED | HALT-and-yield (safe default), or relocate `state_root` outside `CODE_ROOT` when explicitly authorized (e.g. `--auto_relocate_state`); `SNAPSHOT_ROOT` path must not contain `/workspace/` | `mantis-patch` state-vs-code guard misfires; Block A step 3 confuses SNAPSHOT- vs STATE-relative paths | A:3, D:3; INV-5 | `active_snapshot.root`, `snapshot_root`, `state_root` |
419| 2 | Stale active_snapshot (`active_snapshot.pass != state.pass_number`) | PINNED → STOP or HALT-degrade | Block D step 0: handles same-pass re-entry only; if dir missing → STOP, yield to user | Block A step 2 sentinel may still MATCH (dir retained); CURRENT-PASS CHECK (`active_snapshot.pass == state.pass_number`) required: mismatch → STOP or HALT-degrade (Block B NOT_MATCHED, no authoritative verdicts) | A:2, D:0, B; INV-1, INV-3, INV-4, INV-6 | `active_snapshot.{root, snapshot_id, snapshot_pinned, pass}`, `state.pass_number`, `discovery_commit` |
420| 3 | Pin failure | HALT | Block D step 2/4: skip copy on ENOSPC/error → step 5b; still write `active_snapshot` + pass roots | Authoritative verdicts forbidden; Block B always NOT_MATCHED; reproduce `not_attempted`; patch `VERIFICATION_INCOMPLETE` | D:2, D:4, D:5b; INV-1, INV-2, INV-6 | `active_snapshot.{snapshot_id, snapshot_pinned}` |
421| 4 | Patched shadows | PINNED (pass); `--snapshot_pinned=false` arg | Pass `--target_root=<PATCHED_SHADOW_ROOT>` + `--snapshot_pinned=false` to reattack sub-agent | Block A step 1a: `CODE_ROOT=--target_root` (authoritative); step 2 sentinel SKIPPED (sentinel-EXEMPT) | A:1a, A:2; INV-4 | `target_root`, `snapshot_pinned` (arg), `snapshot_root`, `discovery_commit` |
422| 5 | Different-snapshot duplicate candidates | PINNED | No special action — both passes pinned correctly; dedupe handles it | Block B pairwise: `discovery_commit` differs → NOT_MATCHED → keep ACTIVE + `possible_duplicate_of`; POSSIBLE REGRESSION if archived was RESOLVED | B; INV-3, INV-6 | `discovery_commit`, `possible_duplicate_of`, `status`, `patch_status` |
423| 6 | Absent sink evidence | Any | No special action — Block F is a stage-level mechanical gate | Block F: evidence absent (build error, exit 127, sink unreached) → `not_attempted` (retry-eligible), NEVER `failed_to_reproduce`; HALT ceiling additionally forces `not_attempted` | F; INV-2, INV-6 | `repro_status`, `reattack_status`, `repro_hints` |
424
425**Per-scenario detail:**
426
427**1. Colocated state** (`state_root` nested inside `CODE_ROOT` / snapshot root)
428— The pinned `SNAPSHOT_ROOT` must live under
429`<state_root>/.mantis_snapshots/pass_<N>` (or a clean-VCS worktree/archive), and
430its path **must not** contain the segment `/workspace/` — otherwise
431`mantis-patch`'s state-vs-code path guard misfires (state files appear to be
432"under `CODE_ROOT`"). If `state_root` itself is inside `CODE_ROOT`, the harness
433must HALT-and-yield (safe default) or, when explicitly authorized (e.g.
434`--auto_relocate_state`), relocate it outside the snapshot before pinning. Block
435A step 3 distinguishes SNAPSHOT-RELATIVE path fields (read under `CODE_ROOT`)
436from STATE-RELATIVE fields (read under `state_root/workspace`, never prefixed
437with `CODE_ROOT`); colocation breaks this separation.
438
439**2. Stale active_snapshot** (`active_snapshot.pass != state.pass_number` —
440`active_snapshot` was preserved across the Stage 15 pass increment) — Block D
441step 0 (crash-resume) handles only the SAME-pass re-entry case
442(`active_snapshot.pass == N` → reuse). It does NOT catch a stale snapshot
443carried across the Stage 15 pass increment, because Stage 15 deliberately
444preserves `active_snapshot` while bumping `pass_number` (see Stage 15). Two
445sub-cases:
446
447(a) The prior snapshot dir is now MISSING: Block D step 0 STOPs and yields to
448the user (never re-pin to a possibly-drifted live tree). (b) The prior snapshot
449dir still EXISTS (default keep-2 retention) and its sentinel matches the
450preserved `active_snapshot.snapshot_id`: Block A step 2 sentinel check SUCCEEDS
451(it only compares the sentinel file to `SNAPSHOT_ID`, not to the current pass).
452Block B's pairwise `discovery_commit` check would MATCH a carried-forward
453finding against a new finding stamped with the same stale `SNAPSHOT_ID`,
454silently dropping it as `DUPLICATE` — a false authoritative verdict.
455
456To prevent (b), the HARNESS MUST guarantee that
457`active_snapshot.pass == state.pass_number` before any consumer stage reads it.
458The reference harness (`mantis-meta-agent`) satisfies this by re-pinning every
459pass (Block D step 0 sees `active_snapshot.pass != N` → re-pins → refreshes
460`active_snapshot.pass` before any stage runs), so sub-case (b) never fires
461there. A custom harness that preserves `active_snapshot` across the Stage 15
462pass increment WITHOUT re-pinning MUST either (a) re-pin every pass (the
463reference behavior), or (b) inject an equivalent pre-stage gate that refreshes
464`active_snapshot.pass` or clears `active_snapshot` entirely before invoking
465stages. Stages CANNOT self-detect this staleness via Block B (which is
466`snapshot_id`-only, not `pass`-aware): a carried-forward finding and a new
467finding stamped with the same stale `SNAPSHOT_ID` will MATCH in Block B despite
468the snapshot being stale. The `active_snapshot.pass` field is defined in
469`schema.json` `#/$defs/state/active_snapshot/pass` for exactly this check. The
470harness's Block D step 0 reuse check is NOT a substitute: it only fires on
471same-pass re-entry. (Stages that read `active_snapshot` MAY additionally
472self-check defensively — see each stage's Step 0 sentinel check — but the
473binding guarantee is on the harness.)
474
475**3. Pin failure** (snapshot copy fails — disk full, permissions, too-large
476tree) — Block D step 2 (free-space precheck): compare `du -s` of the live tree
477to `df` free space at `state_root`; if it won't fit → skip copy → step 5b. Block
478D step 4 (failure-tolerant verify): check copy exit status + sanity check (file
479count/size within ~90%); on failure → step 5b (unpinned/HALT). Step 5b:
480`SNAPSHOT_ROOT=<live root>`, `snapshot_pinned=false`,
481`SNAPSHOT_ID="live:"+ISO8601`. The harness still writes `active_snapshot` and
482still passes `--snapshot_root`/`--snapshot_id` to stages so they see the HALT
483signal. Every stage then degrades conservatively: authoritative verdicts
484forbidden (`VERIFIED_SECURE`, `failed_to_reproduce`, `DUPLICATE`,
485`FALSE_POSITIVE`, `NON_VIABLE`); Block B always returns NOT_MATCHED; reproduce
486records `not_attempted`; patch's best attainable is `VERIFICATION_INCOMPLETE`.
487
488**4. Patched shadows** (`--target_root` pointing at a pre-mutated tree;
489sentinel-exempt path 1a in Block A) — `mantis-patch` passes
490`--target_root=<PATCHED_SHADOW_ROOT>` and `--snapshot_pinned=false` to the
491reproduce sub-agent for re-attack verification. Block A step 1a:
492`CODE_ROOT = --target_root` (authoritative override, overrides `--snapshot_root`
493and state fallback). Block A step 2: sentinel check SKIPPED (a `--target_root`
494tree is deliberately mutated and is sentinel-EXEMPT). The
495`--snapshot_pinned=false` argument is the sentinel-exemption, NOT a HALT signal
496— detect HALT by reading STATE (`active_snapshot.snapshot_id` starts with
497`live:`, equivalently `active_snapshot.snapshot_pinned` is `false` in state),
498never from the argument passed on this invocation. The finding's
499`discovery_commit` is unaffected — it retains the pass-level `SNAPSHOT_ID` from
500when it was discovered; only the `--snapshot_pinned=false` argument is local to
501the reattack invocation.
502
503**5. Different-snapshot duplicate candidates** (cross-pass dedupe where
504`discovery_commit` differs — the pairwise Block B NOT_MATCHED path) — Both
505passes pinned correctly; the findings simply come from different snapshots.
506`mantis-dedupe` Block B pairwise check compares the CURRENT finding's
507`discovery_commit` against the ARCHIVED finding's `discovery_commit` (NOT
508against the global `SNAPSHOT_ID`). If they differ → NOT_MATCHED. NOT_MATCHED
509keeps the current finding ACTIVE and sets `possible_duplicate_of` (a soft,
510non-terminal hint — the finding is NOT filtered or trashed). If the archived
511finding was RESOLVED (`patch_status` in {`VERIFIED_SECURE`,
512`MITIGATION_PROPOSED`} OR `status`==`FALSE_POSITIVE` OR
513`production_viability`==`NON_VIABLE`) AND the pair is NOT_MATCHED → POSSIBLE
514REGRESSION: keep ACTIVE, add a history note, never filter (a reverted fix
515re-discovered on new code must never be trashed).
516
517**6. Absent sink evidence** (Block F — PoC compiles but produces no reached-sink
518evidence; `not_attempted` vs `failed_to_reproduce`) — `mantis-reproduce` Block
519F: if EVIDENCE is ABSENT (any compiler/build nonzero exit, exit 127
520command-not-found, exit 2 "No such file", or the sink was never reached) →
521`repro_status = not_attempted` (retry-eligible), STOP. NEVER
522`failed_to_reproduce`. In `--reattack` mode: leave `reattack_status` UNSET with
523a history note "setup_failed" — NEVER `failed_to_bypass`. `failed_to_reproduce`
524is reserved for when the harness PROVABLY reached the vulnerable entrypoint —
525i.e. reached-sink evidence, not setup evidence — but the bug did not fire.
526Reached-sink evidence must originate INSIDE the invoked path or from
527target-produced tracing/backtraces: (a) a PoC script/source harness writes
528`MANTIS_REACHED_ENTRYPOINT` to a sidecar file at the point just before the sink
529call, within its own execution flow (the marker write is part of the invoked
530path, not a pre-launch step); OR (b) for binary/firmware/raw-payload targets,
531the captured crash backtrace or sanitizer trace (ASan/UBSan/MSan/TSan)
532explicitly names the target sink function (target-produced tracing). A marker
533written by an external wrapper BEFORE invoking the target is SETUP EVIDENCE ONLY
534(proves "launch attempted," not "sink reached") and does NOT by itself justify
535`failed_to_reproduce` — treat it as EVIDENCE ABSENT for the decision gate.
536Evidence is recorded in `repro_hints`. In HALT mode, the HALT ceiling
537additionally forces `not_attempted` (no `failed_to_reproduce`), since a negative
538result on an unpinned tree cannot be trusted as authoritative.
539
540______________________________________________________________________
541
542### 6. Semantic Retrieval (RAG) for Large Codebases
543
544For small repositories, the planner can manually scan `workspace/kb/index.md`
545and the researcher can grep for call-sites. At scale (thousands of files, deep
546directory trees, multi-pass campaigns), these approaches miss relevant context
547and waste tokens reading irrelevant files. A semantic retrieval layer lets the
548planner and researcher query for relevant KB entries and code locations without
549reading everything.
550
551Two implementations are supported, sharing the same data contract:
552
553- **Option A (Default — Skill-Based):** A dedicated skill that runs a
554 BM25/TF-IDF helper script over `chunks.jsonl`. Zero external dependencies —
555 works air-gapped, no vector embeddings or vector store required. Optional
556 vector embedding support if available.
557- **Option B (Maximum Scale — MCP-Based):** The harness owns a persistent vector
558 index using vector embeddings, serving persistent `semantic_search_kb` /
559 `semantic_search_code` MCP tools. Better for very large codebases where
560 per-invocation BM25 is too slow.
561
562Both are **opt-in**. The existing skills are not modified; the planner and
563researcher receive runtime instructions to use whichever retrieval mechanism is
564available, falling back to today's manual behavior if neither is present.
565Retrieval results are **coverage HINTs only** — they decide ordering and
566prioritization, never the membership of the audit set. A miss must never cause a
567file, call-site, or investigation to be skipped or dropped.
568
569#### A. Shared Data Contract: `chunks.jsonl`
570
571After Stage 2 (`/mantis-architecture`) completes, chunks are extracted into
572`workspace/kb/chunks.jsonl` (one JSON object per line). The harness can do this
573post-hoc by reading `workspace/kb/*.md`, or the architecture skill can be
574instructed to write it during synthesis as a text-only side effect. Two chunk
575types are produced:
576
5771. **KB chunks** from the existing `workspace/kb/*.md` files:
578
579 ```json
580 {"id": "auth_module:0", "source_file": "workspace/kb/entities/auth_module.md", "entity_type": "entity", "chunk_text": "The auth module handles..."}
581 ```
582
5832. **Code chunks** from `CODE_ROOT` (the pinned snapshot). Each chunk includes
584 the file path and line range so the researcher can request specific files
585 from the snapshot:
586
587 ```json
588 {"id": "src/parser.c:0", "source_file": "src/parser.c", "start_line": 1, "end_line": 80, "chunk_text": "int parse_input(..."}
589 ```
590
591The first line of `chunks.jsonl` is a provenance header recording the
592`SNAPSHOT_ID` the chunks were built against:
593
594```json
595{"_provenance": true, "snapshot_id": "abc123", "kb_snapshot_id": "abc123"}
596```
597
598Before serving queries, check `snapshot_id` in the provenance header against the
599current `SNAPSHOT_ID`; rebuild if they differ. In MODE-OFF (no
600`active_snapshot`), `kb_snapshot_id` is never stamped — skip the index entirely
601and let skills fall back to manual scanning. Never build code chunks from the
602live tree — they must reflect the pinned copy the skills are reading.
603
604#### B. Option A: Skill-Based Retrieval (Default — No Infrastructure)
605
606A dedicated skill reads `chunks.jsonl` and writes+runs a helper script (e.g.
607`workspace/helpers/search_chunks.py`) that performs BM25/TF-IDF similarity
608search. The script is generated by the agent at runtime — no code is shipped
609with the skill (same pattern as `mantis-dedupe`'s `merge_findings.py`). This
610requires zero external dependencies — no embedding model, no vector store, no
611MCP server. It works in air-gapped and VPC-SC environments.
612
613A complete reference blueprint for this skill is available at
614[references/mantis-kb-query.md](references/mantis-kb-query.md). Builders can
615adapt it to their environment. The blueprint includes Block A (Locator
616Resolution), chunk provenance checking, the versioned helper script contract
617(`MANTIS_HELPER_VERSION = 1`), and the JSON output schema.
618
619- **Invocation:** The planner or researcher spawns the skill as a sub-agent with
620 a query string. The skill writes the helper if not already present, runs it,
621 and returns top-K matching chunks as JSON.
622- **Optional embeddings:** If vector embeddings are available, the agent can be
623 instructed to use cosine similarity instead of BM25. This is a runtime
624 configuration toggle, not a different skill.
625- **Snapshot safety:** The skill reads `active_snapshot` from state via Block A
626 (same as every other skill) and checks chunk provenance before serving.
627
628#### C. Option B: MCP-Based Retrieval (For Maximum Scale)
629
630For very large codebases where per-invocation BM25 is too slow, the harness can
631own a persistent vector index using vector embeddings, serving two MCP tools
632(following the same pattern as Guideline 2's Custom MCP for VMs/hardware):
633
634- `semantic_search_kb(query: string) → [{id, source_file, entity_type, chunk_text, score}]`
635 — Searches KB chunks. Returns relevant entity/vulnerability markdown context.
636
637- `semantic_search_code(query: string) → [{file, start_line, end_line, snippet, score}]`
638 — Searches code chunks from the pinned snapshot. Returns relevant code
639 locations.
640
641The harness manages the vector index lifecycle: build from `chunks.jsonl` (or
642directly from `CODE_ROOT`), rebuild when `SNAPSHOT_ID` changes, and handle
643freshness checks. In HALT mode, serve with a `STALE` flag or refuse. In
644MODE-OFF, skip entirely.
645
646#### D. Per-Skill Augmentation Guidance
647
648When a retrieval mechanism (skill or MCP) is available, instruct the following
649skills to use it. These are runtime instructions passed by the harness or
650meta-agent when invoking the skill — the skill files themselves are not
651modified:
652
653- **mantis-architecture:** No changes needed. The harness chunks the existing
654 `workspace/kb/*.md` files after the architect completes Stage 2. If the
655 builder prefers, they may instruct the architect to also write
656 `workspace/kb/chunks.jsonl` during synthesis (Step 3) as a text-only side
657 effect — but this is optional, since the harness can extract chunks post-hoc.
658
659- **mantis-plan:** If a retrieval mechanism is available, instruct the planner
660 to use it to discover `kb_references` for each investigation instead of only
661 manually scanning `workspace/kb/index.md`. For each investigation, query with
662 the investigation title and target file names, then add the top-K matching KB
663 entity/vulnerability files to the `kb_references` array. Manual scanning of
664 `index.md` remains the fallback when no mechanism is available.
665
666- **mantis-researcher:** If a retrieval mechanism is available, instruct Wave 1
667 sub-agents to use it to PRIORITIZE relevant call-sites and cross-module data
668 flows into sinks (e.g., "where does untrusted input reach `memcpy` in the
669 parser module"). Semantic search SUPPLEMENTS grep as a ranking HINT ONLY — it
670 decides ORDER, never MEMBERSHIP of the audit set. It MUST NEVER replace the
671 exhaustive Step-3 call-site sweep; every call-site or data-flow that a full
672 grep would reach must still be audited whether or not it ranks in top-K. Audit
673 the union of grep results and semantic search results. The researcher's
674 existing Wave 1/Wave 2 structure is unchanged.
675
676#### E. Snapshot Safety
677
678The retrieval index — whether served by the skill or the harness — is a **cache
679of the pinned snapshot**, never a live view:
680
681- Build code chunks from `CODE_ROOT` (the pinned snapshot), not the live tree.
682- Rebuild when `SNAPSHOT_ID` changes (new pass, new pin).
683- In HALT mode (`snapshot_pinned=false`), serve results with a `STALE` flag or
684 refuse to serve — same conservative degradation as every other stage.
685- In MODE-OFF, skip the index entirely.
686
687______________________________________________________________________
688
689### 7. Embedding-Based Deduplication Pre-Filtering
690
691When the pipeline runs many passes over a large codebase, the deduplicator
692(`/mantis-dedupe`) must compare each current finding against every archived
693finding — an O(n×m) comparison performed by an LLM reading summaries. At scale
694(hundreds of findings across many passes), this is token-expensive and slow.
695
696The harness can use embeddings as a **fast pre-filter** to reduce the candidate
697space before invoking `/mantis-dedupe`. The skill's existing deterministic
698matching (`code_paths` + `title` + `discovery_commit`) remains the **sole
699authority** for hard dedup decisions.
700
701**This is entirely harness-side and opt-in.** `/mantis-dedupe` is not modified.
702
703#### A. How It Works
704
7051. **Harness Action:** Reads all `workspace/findings/*.json` (current) and
706 `workspace/archive/findings_pass_*/*.json` (archived). For each finding,
707 computes an embedding from a normalized text representation (e.g.,
708 `title + description + first code_paths entry with line stripped`).
709
7102. **Harness Action:** Computes pairwise cosine similarity between current and
711 archived findings. Surfaces candidate pairs above a configurable threshold
712 (e.g., 0.85).
713
7143. **Harness Action:** Writes a candidate-pairs manifest (e.g.,
715 `workspace/helpers/dedup_candidates.json`) containing the UUID pairs and
716 similarity scores.
717
7184. **mantis-dedupe invocation:** The harness invokes `/mantis-dedupe` as usual.
719 If the candidate manifest exists, instruct the skill to read it and
720 prioritize those pairs for the LLM's pairwise comparison, instead of
721 comparing every finding against every archived finding. **All existing
722 deterministic matching rules apply unchanged** — the manifest only narrows
723 the search space.
724
725#### B. Safety Guardrails
726
727- **Pre-filter only, never the decision.** The embedding similarity score can
728 never cause a `DUPLICATE` verdict, a trash move, or a `possible_duplicate_of`
729 assignment. Only the skill's existing `code_paths`, `title`, and
730 `discovery_commit` checks can do that. A miss in the pre-filter only
731 over-retains a duplicate (safe — the LLM sees it and skips it); it never
732 under-retains (never drops a real duplicate).
733
734- **No false negatives.** The threshold should be set low enough (e.g., 0.75) to
735 avoid missing true duplicates. Better to surface too many candidates than to
736 miss a real one — the LLM and deterministic matching will filter false
737 positives.
738
739- **Fallback on failure.** If the embedding computation is unavailable, the
740 candidate manifest is absent, or any error occurs, `/mantis-dedupe` falls back
741 to its existing O(n×m) comparison. The skill must not stop or error if the
742 manifest is missing.
743
744- **Snapshot awareness.** The harness should not compute embeddings across
745 different `discovery_commit` values without flagging them as cross-snapshot
746 candidates — the skill's Block B pairwise check will handle the final
747 MATCHED/NOT_MATCHED decision.
748
749#### C. Shared Infrastructure
750
751If the builder also implements Guideline 6 (Semantic Retrieval), reuse the same
752vector embedding infrastructure for finding embeddings. The finding embedding is
753a different payload (finding JSON, not KB chunks) but the same embedding
754capability can serve both.
755
756#### D. Enterprise Vector Architecture & Cloud Scalability
757
758For enterprise deployments tracking hundreds of thousands of vulnerabilities
759across distributed codebases and multi-tenant audit pipelines, SQLite vector
760storage can be swapped for managed cloud vector engines:
761
762- **Cloud Spanner Vector Search:**
763 - *Best For*: Mission-critical enterprise pipelines requiring global ACID
764 consistency, 99.999% availability, and synchronized finding lineage
765 tracking.
766 - *Architecture*: Store findings and `lineage_vectors` in Spanner with
767 `ARRAY<FLOAT32>` vector columns. Utilize `COSINE_DISTANCE` with KNN search
768 indexes for real-time finding deduplication during parallel scan passes.
769- **Cloud SQL for PostgreSQL (`pgvector`):**
770 - *Best For*: Standard enterprise relational backends needing ACID compliance
771 with low operational complexity.
772 - *Architecture*: Enable the `vector` extension. Store finding embeddings in a
773 `vector(768)` or `vector(256)` column with HNSW (`vector_cosine_ops`) or
774 IVFFlat indexes for sub-millisecond similarity queries.
775- **AlloyDB for PostgreSQL:**
776 - *Best For*: High-throughput multi-agent audit farms running concurrent
777 research waves.
778 - *Architecture*: Leverage AlloyDB's columnar vector engine and ScaNN-based
779 approximate nearest neighbor indexing for up to 10x faster vector queries
780 over standard PostgreSQL.
781- **BigQuery Vector Search:**
782 - *Best For*: Batch analytical processing, organization-wide threat
783 intelligence clustering, and cross-campaign vulnerability lineage analysis.
784 - *Architecture*: Ingest finding embeddings into BigQuery and execute
785 `VECTOR_SEARCH(TABLE findings, 'embedding', ...)` with `COSINE` distance for
786 serverless batch deduplication and historical regression analytics.
787
788______________________________________________________________________
789
790### 8. SAST Seeding (External Tool Ingestion)
791
792Mantis's discovery engine is 100% LLM-generative (grep swarm + reasoning). A
793weak LLM can structurally under-detect whole-program taint classes (injection,
794path traversal, deserialization, UAF, format-string) that mature SAST tools
795(CodeQL, Semgrep-taint) encode as interprocedural queries. A SAST seeding
796adapter ingests external tool findings as `PROVISIONALLY_VALID` /
797`NEEDS_RESEARCH` candidates that must earn their verdict through the unchanged
798downstream gates. This is purely additive (INV-2-strengthening) — it expands
799detection breadth without weakening any verification gate.
800
801This follows exactly the RAG pattern from Guideline 6: opt-in, default off,
802provenance-tracked, snapshot-aware, fallback on failure.
803
804**Platform-agnostic IR (not SARIF):** Rather than tying the adapter to SARIF (a
805complex, tool-specific format), the adapter consumes a minimal JSONL
806intermediate representation (IR). Any SAST tool's output (SARIF, Semgrep JSON,
807Bandit JSON, etc.) is converted to this IR by a thin wrapper. This maximizes
808platform agnosticism — the adapter works with any tool that can produce the
809simple JSONL format.
810
811Two implementations are supported, sharing the same data contract:
812
813- **Option A (Default — Skill-Based):** A dedicated skill (`/mantis-sast-seed`)
814 reads the IR and uses LLM reasoning to normalize findings into mantis finding
815 JSONs. The LLM reads actual source code at each reported location under
816 CODE_ROOT to verify the finding and enrich the description with root-cause
817 analysis. Zero external dependencies — works air-gapped.
818- **Option B (Maximum Control — Harness-Based):** The harness directly
819 normalizes SAST output into finding JSONs using deterministic code (e.g., a
820 SARIF-to-finding converter script), bypassing the LLM for the normalization
821 step. The harness writes finding JSONs to `workspace/findings/` before
822 invoking `/mantis-dedupe`.
823
824Both are **opt-in**. The existing skills are not modified. Seeded findings enter
825`workspace/findings/` alongside researcher findings and flow through the
826unchanged downstream gates (dedupe -> review -> critic -> reproduce -> patch ->
827calibrate).
828
829A complete reference blueprint is available at
830[references/mantis-sast-seed.md](references/mantis-sast-seed.md).
831
832#### A. Shared Data Contract: `sast_findings.jsonl`
833
834The IR is a JSONL file at `workspace/sast_findings.jsonl` (STATE-RELATIVE). It
835follows the same provenance-header pattern as `chunks.jsonl` (Guideline 6A).
836
837**Line 1 — Provenance header:**
838
839```json
840{"_provenance": true, "scan_snapshot_id": "abc123def456", "tool": "codeql", "tool_version": "2.15.0", "scan_timestamp": "2026-07-22T10:00:00Z"}
841```
842
843- `scan_snapshot_id`: The SNAPSHOT_ID the scan was run against. This is the
844 **primary provenance anchor** — compared byte-for-byte to the current pass
845 `SNAPSHOT_ID` (same comparison as Block B). If the harness ran the SAST tool
846 against the pinned CODE_ROOT, it sets this to `SNAPSHOT_ID`.
847- `tool`: Tool name (e.g., `codeql`, `semgrep`, `bandit`). Informational.
848- `tool_version`: Tool version. Informational.
849- `scan_timestamp`: ISO 8601. Informational.
850
851**Lines 2+ — One finding per line:**
852
853```json
854{"rule_id": "cpp/sql-injection", "rule_name": "SQL injection", "cwe": "CWE-89", "severity": "HIGH", "code_paths": ["src/db/query.c:42"], "message": "User input flows into SQL query without sanitization"}
855```
856
857| Field | Type | Required | Description |
858| ------------ | ------ | -------- | -------------------------------------------------- |
859| `rule_id` | string | Yes | Tool-specific rule identifier |
860| `severity` | string | Yes | `CRITICAL`, `HIGH`, `MEDIUM`, `LOW` (Mantis scale) |
861| `code_paths` | array | Yes | Array of `"file:line"` strings (SNAPSHOT-RELATIVE) |
862| `message` | string | Yes | Original SAST finding message |
863| `rule_name` | string | No | Human-readable rule name |
864| `cwe` | string | No | CWE identifier (e.g., `CWE-89`) |
865
866**IR conversion (harness responsibility):** The harness converts SAST tool
867output to this IR before invoking the adapter. Conversion examples:
868
869- SARIF: extract `ruleId`, map `level` to severity (`error`->`HIGH`,
870 `warning`->`MEDIUM`, `note`->`LOW`), extract `locations` to `code_paths`, copy
871 `message.text`.
872- Semgrep JSON: extract `check_id` as `rule_id`, map `extra.severity` to Mantis
873 severity, extract `path:start.line` as `code_paths`, copy `extra.message`.
874
875#### B. Option A: Skill-Based Ingestion (Default — No Infrastructure)
876
877A dedicated skill reads `sast_findings.jsonl`, applies allow-listing, verifies
878provenance, computes `signature`/`lineage_id`/`discovery_commit`, and writes
879finding JSONs. The LLM reads actual source code at each reported location under
880CODE_ROOT to verify the finding and enrich the description.
881
882- **Invocation:** The harness invokes `/mantis-sast-seed` with
883 `--snapshot_root`/`--snapshot_id`/`--state_root` after Stage 6 (Research) and
884 before Stage 7 (Dedupe). The seeded findings land in `workspace/findings/`
885 before `/mantis-dedupe` runs.
886- **Inert until wired:** If `sast_findings.jsonl` is absent, the skill outputs
887 nothing and notifies the caller. It never fails — it simply returns empty.
888- **Anti-hallucination:** The LLM MUST read the actual code at each reported
889 location before writing the finding JSON. It MUST NOT invent findings not
890 present in the SAST output. The `sast_provenance.line_verified` field records
891 whether verification succeeded.
892- **Snapshot safety:** The skill reads `active_snapshot` from state via Block A
893 and stamps `discovery_commit` only when the snapshot is pinned and the
894 finding's location is verified under CODE_ROOT.
895
896#### C. Option B: Harness-Based Ingestion (For Deterministic Ingestion)
897
898For maximum determinism, the harness can normalize SAST output into finding
899JSONs using deterministic code. This bypasses the LLM for the normalization
900step:
901
9021. **Harness Action:** Reads the SAST tool output, parses it deterministically,
903 and writes finding JSONs to `workspace/findings/`.
9042. **Harness Action:** Stamps `discovery_commit` only if the scan provably ran
905 against the pinned CODE_ROOT.
9063. **Harness Action:** Applies allow-listing filters.
9074. **mantis-dedupe invocation:** Proceeds as usual — the seeded findings are
908 indistinguishable from researcher findings.
909
910#### D. Provenance Verification
911
912The adapter stamps `discovery_commit` ONLY if the scan provably ran against a
913line-identical pinned CODE_ROOT. Verification ladder:
914
9151. Read `active_snapshot` from `workspace/.mantis_state.json` (via Block A). If
916 absent, this is MODE-OFF — skip to step 5.
9172. If `snapshot_pinned` is false, this is HALT — skip to step 6.
9183. Read the IR provenance header's `scan_snapshot_id`.
9194. **VERIFIED**: `scan_snapshot_id` present AND exactly equals `SNAPSHOT_ID`.
920 Stamp `discovery_commit = SNAPSHOT_ID`. Status = `PROVISIONALLY_VALID`.
9215. **MODE-OFF**: `active_snapshot` absent. OMIT `discovery_commit`. Status =
922 `PROVISIONALLY_VALID` (MODE-OFF permits all verdicts).
9236. **HALT**: `snapshot_pinned` is false. OMIT `discovery_commit`. Status =
924 `NEEDS_RESEARCH`.
9257. **DRIFT**: `scan_snapshot_id` present but differs. OMIT `discovery_commit`
926 entirely. Status = `NEEDS_RESEARCH`.
9278. **UNVERIFIED**: `scan_snapshot_id` absent. OMIT `discovery_commit`. Status =
928 `NEEDS_RESEARCH`.
929
930**Line-existence verification** (additional check when VERIFIED or DRIFT and
931CODE_ROOT is resolved): For each finding, verify each `code_paths` entry: strip
932trailing `:line`, check file exists under CODE_ROOT, check line number is within
933file's line count. If file missing or line out of range -> downgrade to DRIFT
934(omit `discovery_commit`, status = `NEEDS_RESEARCH`).
935
936This is the exact same "UNTRUSTED-IF-ABSENT" pattern as `discovery_commit`
937(schema.json) and `signature`/`lineage_id`.
938
939#### E. Allow-Listing and Noise Control
940
941`workspace/sast_allowlist.json` (optional config file, STATE-RELATIVE):
942
943```json
944{
945 "enabled": true,
946 "severity_filter": ["CRITICAL", "HIGH"],
947 "cwe_allowlist": {
948 "enabled": true,
949 "cwes": ["CWE-89", "CWE-78", "CWE-79", "CWE-22", "CWE-787", "CWE-416", "CWE-502"]
950 },
951 "rule_allowlist": {
952 "enabled": false,
953 "rules": []
954 },
955 "per_rule_cap": 5,
956 "total_cap": 50
957}
958```
959
960If the config file is absent, defaults apply: CRITICAL+HIGH only,
961`per_rule_cap=5`, `total_cap=50`.
962
963**How allow-listing protects the retry cap:** The reproduce stage
964(`/mantis-reproduce`) has a hard ceiling of 6 attempts per finding (absolute,
965never reset). If 1000 SAST findings are seeded without filtering, the reproduce
966stage would need up to 6000 attempts — starving the retry budget. The
967allow-listing chain (severity filter -> CWE/rule filters -> per-rule cap ->
968total cap) ensures only a bounded, high-signal set of candidates enters the
969pipeline. The review (13-rule negative filter) and critic (production viability)
970stages further filter before reproduce runs.
971
972#### F. Per-Skill Augmentation Guidance
973
974When the SAST seed skill is available, instruct the harness to invoke it between
975Stage 6 (Research) and Stage 7 (Dedupe). These are runtime instructions passed
976by the harness — the skill files themselves are not modified:
977
978- **mantis-meta-agent:** Invoke `/mantis-sast-seed` with
979 `--snapshot_root`/`--snapshot_id`/`--state_root` after `/mantis-researcher`
980 completes and before `/mantis-dedupe`.
981- **mantis-dedupe:** No changes needed. Seeded findings are in
982 `workspace/findings/` alongside researcher findings. Dedupe processes them
983 identically (signature-based matching, Block B pairwise check).
984- **mantis-review:** No changes needed. The 13-rule negative filter applies to
985 seeded findings identically.
986- **mantis-report:** No changes needed. The `sast_provenance` field is
987 informational and can be displayed in reports.
988
989#### G. Snapshot Safety
990
991The SAST seed adapter follows Block A (Locator Resolution) exactly like every
992other code-reading skill:
993
994- Resolve CODE_ROOT from `--snapshot_root` / state `active_snapshot`.
995- Honor the sentinel check (Block A step 2).
996- Read source files under CODE_ROOT (SNAPSHOT-RELATIVE) for line-existence
997 verification.
998- Write findings under `state_root/workspace/findings/` (STATE-RELATIVE).
999- Never write under CODE_ROOT when pinned.
1000- In MODE-OFF: proceed without `discovery_commit`. All verdicts permitted.
1001- In HALT: omit `discovery_commit`. Seeded findings get `NEEDS_RESEARCH`.
1002- In PINNED: verify each finding's location under CODE_ROOT, stamp
1003 `discovery_commit = SNAPSHOT_ID` if verified.
1004
1005The SAST tool output itself is NOT snapshot-aware — it may have been produced
1006against a different tree. The adapter's provenance verification is what bridges
1007the gap: it re-grounds each finding against the pinned CODE_ROOT before stamping
1008`discovery_commit`.
1009
1010#### H. Safety Guardrails
1011
1012- **Purely additive.** Seeded findings are `PROVISIONALLY_VALID` /
1013 `NEEDS_RESEARCH` candidates. They must pass through the unchanged downstream
1014 gates: dedupe (Block B), review (13-rule filter), critic (viability),
1015 reproduce (Block F reached-sink evidence + HALT ceiling), patch (Block G
1016 re-attack), calibrate (sanity caps). No gate is weakened.
1017- **No false `VERIFIED_SECURE`.** Seeded findings start at `PROVISIONALLY_VALID`
1018 — they can never reach `VERIFIED_SECURE` without passing through the full
1019 patch + re-attack pipeline.
1020- **No false `failed_to_reproduce`.** Seeded findings that reach reproduce are
1021 subject to the same Block F evidence gate and HALT ceiling.
1022- **No dropped regression.** If a SAST-seeded finding matches an archived
1023 finding with a different `discovery_commit`, Block B returns NOT_MATCHED ->
1024 `possible_duplicate_of` -> finding stays active.
1025- **Fail-safe on missing data.** If `sast_findings.jsonl` is absent, no findings
1026 written. If a finding can't be verified, `NEEDS_RESEARCH`.
1027- **No existing skills modified.** The adapter writes findings to
1028 `workspace/findings/` before `/mantis-dedupe` runs.
1029- **`sast_provenance` is informational only.** It does not affect any safety-
1030 critical invariant, gate, or verdict. The finding's `discovery_commit` is the
1031 field that governs Block B snapshot matching.
1032
1033______________________________________________________________________
1034
1035### 9. Structural Code Index (AST-Level Context)
1036
1037For small repositories, the researcher can grep for call-sites and the planner
1038can infer dependencies. At scale, grep-based call-site discovery is unreliable
1039(misses indirect calls, cannot distinguish calls from comments/strings, no
1040function boundary awareness). A structural code index provides AST-level context
1041(function boundaries, call graphs, symbol tables) to improve LLM reasoning
1042quality during discovery.
1043
1044This follows the RAG pattern from Guideline 6: optional, provenance-tracked,
1045snapshot-aware, fallback on failure. The structural index is a **coverage HINT
1046only** — it decides ordering and prioritization, never the membership of the
1047audit set. A miss must never cause a file, call-site, or investigation to be
1048skipped or dropped.
1049
1050The full specification — including the manifest schema, SQLite serving store,
1051capability-based per-partition backend selection, canonical symbol IDs, query
1052interface, baseline-plus-delta overlay, deterministic partial coverage, and
1053safety guardrails — lives in a single source of truth:
1054
1055**→ [../mantis-structural-index/SKILL.md](../mantis-structural-index/SKILL.md)**
1056
1057A thin reference blueprint is at
1058[references/mantis-structural-index.md](references/mantis-structural-index.md).
1059
1060Two implementations are supported, sharing the same query contract:
1061
1062- **Option A (Default — Skill-Based):** The `mantis-structural-index` skill
1063 generates and runs helper scripts (`build_structural_index.py` and
1064 `query_structural_index.py`, both `# MANTIS_HELPER_VERSION = 5`) using
1065 capability-based per-partition backend selection, degrading to grep.
1066- **Option B (Maximum Power — MCP-Based):** The harness owns a persistent
1067 structural index serving `find_callers(symbol)`, `find_callees(function)`,
1068 `get_function_boundary(file, line)` MCP tools, backed by the same SQLite
1069 serving store.
1070
1071Both are **optional**. A non-conformant harness simply skips the structural
1072index stage. The structural index supplements grep as a ranking HINT ONLY — it
1073decides ORDER, never MEMBERSHIP of the audit set.
1074
1075Consumers MUST use the query interface (`query_structural_index.py`) rather than
1076filtering JSONL directly. The query interface provides bounded results,
1077pagination, explicit name resolution, precision/backend metadata, and coverage
1078on empty results.
1079
1080#### D. Per-Skill Augmentation Guidance
1081
1082When a structural index is available, instruct the following skills to use it.
1083These are the consumption contract — runtime instructions passed by the harness.
1084The structural index is a HINT-only enhancement; skills that do not use it
1085behave exactly as they do today:
1086
1087- **mantis-architecture:** Optionally read the pre-built structural index (built
1088 by `mantis-structural-index` at Stage 0.5) during KB synthesis,
1089 cross-referencing it with `dependencies.json`.
1090- **mantis-plan:** Use the structural index for function-level dependency
1091 fan-out (more precise than file-level `dependencies.json`).
1092- **mantis-researcher:** Wave 1 sub-agents use `find_callers()` to SUPPLEMENT
1093 grep as a ranking HINT for call-site discovery — it decides ORDER, never
1094 MEMBERSHIP. It MUST NEVER replace the exhaustive Step-3 call-site sweep; every
1095 call-site that a full grep would reach must still be audited whether or not it
1096 ranks in the structural index. Audit the union of grep results and structural
1097 index results. Wave 2 deep auditors use `get_function_boundary()` to start
1098 with the enclosing function, expanding to callers/callees/file as needed for
1099 cross-function context. The researcher's existing Wave 1/Wave 2 structure is
1100 unchanged.
1101
1102#### E. Integration with RAG (Guideline 6)
1103
1104- Structural index entries can be added to `chunks.jsonl` as
1105 `entity_type: "structural"` chunks.
1106- Code chunks can become function-boundary-aligned (instead of dumb line-range
1107 slices) by using function boundaries from the structural index.
1108- The structural index and the RAG index can share the same vector embedding
1109 infrastructure if both are implemented.
1110
1111______________________________________________________________________
1112
1113### 10. Tiered Iterative Reproduction & Multi-Conversation Retry Strategy
1114
1115For complex services, attempting a single-shot reproduction directly against a
1116full sandboxed service often suffers from high search entropy, brittle
1117configuration, and hard-to-debug failures. A tiered strategy breaks reproduction
1118into incremental milestones, while an inter-conversation retry architecture
1119prevents reasoning deadlocks and context bloat.
1120
1121#### A. Orchestration & Inter-Conversation Retries
1122
11231. **Intra-Conversation Retries (Local Agent Trajectory):**
1124
1125 - The active subagent conversation retries 2–3 times locally within its
1126 context window to adjust parameters, fix setup bugs, or refine payloads.
1127
11282. **Inter-Conversation Retries (Fresh Context + Accumulated Artifacts):**
1129
1130 - **Trigger:** If intra-conversation retries fail to reach Tier 3
1131 (`reproduced`), the orchestrator terminates the stalled conversation and
1132 launches a **new subagent conversation** (a fresh context window).
1133 - **Context Provisioning:** The orchestrator populates the new prompt with
1134 structured attempt data from
1135 `state_root/workspace/archive/.repro_attempts.json` and trajectory
1136 learnings from `workspace/learnings.jsonl` (e.g., *"Attempt 1 failed due to
1137 missing auth header X; Attempt 2 proved parser strips unescaped quotes"*).
1138 - **Benefit:** Eliminates context bloat and reasoning inertia ("hallucination
1139 traps"), enabling a fresh agent to solve the problem using prior empirical
1140 observations without repeating past mistakes.
1141
11423. **Attempt Cap Accounting & Arithmetic:**
1143
1144 - The orchestrator maintains
1145 `state_root/workspace/archive/.repro_attempts.json`.
1146 - **Tier 3 Increment Only:** Only Tier-3 full sandboxed service executions
1147 (or full end-to-end reproducer runs) increment the per-finding attempt
1148 counter toward the absolute hard ceiling of 6.
1149 - **Stepping-Stone Sub-Budget:** Internal Tier-1 and Tier-2 trial runs are
1150 bounded by a local sub-budget (max 3 trial executions per conversation) and
1151 do not consume the absolute 6-attempt cap.
1152
1153#### B. Harness-Enforced Tier 4 (Staging & Live Pre-Production Execution)
1154
1155To prevent un-gated exploit execution on live infrastructure, Tier 4 (Staging /
1156Pre-Production verification) is **strictly owned and enforced by the
1157programmatic orchestrator harness**, never by LLM discretion:
1158
1159- **Sandbox Boundary:** `mantis-reproduce` executes strictly within isolated
1160 local sandboxes (Tiers 1–3) ending at Tier 3 (`reproduced` /
1161 `failed_to_reproduce`).
1162- **Deterministic Gate:** The harness intercepts a Tier 3 `reproduced` verdict.
1163 If live/staging validation (Tier 4) is configured, the harness **MUST NOT**
1164 automatically invoke remote execution. It must enforce a **programmatic
1165 Human-in-the-Loop gate**:
1166 - Prompt the human operator for explicit interactive confirmation, OR
1167 - Require a cryptographically signed approval token / authorization callback.
1168- **Fail-Closed Default:** If human approval is missing or denied, Tier 4 is
1169 skipped and the Tier 3 sandboxed verdict remains authoritative.
1170
1171#### C. Local Ingress / Middlebox Edge Annotation (Post-Tier-3 Sandbox Verification)
1172
1173To eliminate false-positive findings caused by default edge filters ("Works on
1174localhost:8080, but dies at the WAF/proxy"), the orchestrator can optionally
1175execute a Tier 3 PoC through a local reverse proxy or API gateway (e.g. NGINX,
1176Envoy, ModSecurity) running inside the local sandbox:
1177
1178- **Annotation Only:** A payload blocked by a local middlebox MUST NOT downgrade
1179 a Tier 3 `reproduced` verdict to `failed_to_reproduce`.
1180- **Critic Integration:** The harness records `ingress_blocked: true` (or edge
1181 filter details) in the finding's `repro_hints` or history. This provides
1182 empirical evidence for `/mantis-critic` to classify `production_viability` as
1183 `"CONDITIONAL_VIABLE"` (mitigated by default edge proxy configuration).