Research Ingest

Research Ingest

Turn an external source (URL or raw/research/ file) into durable wiki knowledge. Synthesise into existing technique/tool pages; do NOT create one page per source. Full workflow: docs/workflows.md (Research ingest / Git repo ingest).

Procedure

1. Fetch the source.

   • URL: defuddle parse <url> --md for clean markdown (install: npm i -g defuddle); fall back to the WebFetch tool.
   • Local: read the file under raw/research/. GitHub repo: clone via WSL only (wsl -d kali-linux -u kali -- git clone <url> /home/kali/<name>), then read source/README.

2. Find the home page. qmd_query "<technique/CVE class>" via wiki-search MCP -> identify the technique/tool page that should hold this. One class = one page.

3. Dedup (skip rule). Read only the target page's frontmatter. If the ingest slug is already in sources:, STOP - already ingested. Otherwise continue.

4. Synthesise the new payload / bypass / CVE chain / detail into the page's existing sections (Methodology, Key payloads, Bypasses and variants). Add concrete commands in code blocks. Reconstruct any image into a code block - never embed images. No em-dashes.

5. Tool page if a new standalone tool appears: create/update wiki/tools/<tool>.md.

6. Record the source. Add the slug to the page's sources: list; bump date_updated. Register URL/path + slug in raw/manifest.md.

7. Re-index + log. python3 scripts/gen_index.py (catalog) and, after a bulk session, qmd update (search index). Append a one-line entry to session/log.md. Run python3 scripts/lint-wiki.py -q - must be clean.

Guardrails

• Generic only. Wiki holds reusable methodology; client/engagement specifics stay in targets/<eng>/. Reusable default creds -> wiki/cheatsheets/default-credentials.md; reusable request patterns -> api-request-findings.md.
• Prefer enriching an existing page over creating a new one. New page only for a genuinely new technique/tool class.

Report: page(s) updated + slug added + lint status.