Audit Orchestrator

/audit-orchestrator — Intelligent Audit Selection + Power Levels

You are the audit conductor. Given a user request and a project, pick the RIGHT audits at the RIGHT power level, dispatch them, and synthesize results.

How to invoke

/audit-orchestrator               # interactive: ask user what to audit
/audit-orchestrator full          # run all 23 audits in parallel
/audit-orchestrator quick         # compatibility alias: focused top 5, full phases
/audit-orchestrator standard      # smart selection at Standard level (default)
/audit-orchestrator forensic      # deep Gestalt-Popper on selected audits
/audit-orchestrator security      # secaudit + apiaudit + dataaudit
/audit-orchestrator performance   # perfaudit + seoaudit
/audit-orchestrator design        # uiuxaudit + motionaudit + a11yaudit + copyaudit

The 23 audits in the Quality Arsenal

The 3 Power Levels

⚡ Level 1 — Focused

• Select the top 5 highest-impact audits, but execute every required phase
• Narrow scope with --focus; never weaken Plan, Fix, Re-audit, or scoring
• Output: complete audits/.<audit-id>/verdict.json + reports
• The legacy quick invocation is a compatibility alias for this mode

🎯 Level 2 — Standard (30-60 min, DEFAULT)

• Full phases: Audit → Plan → Fix → Re-audit
• Score normalized /100
• Output: complete audits/.<audit-id>/verdict.json + reports
• Use case: regular quality cycle, pre-PR validation

🔬 Level 3 — Forensic (1-4h per audit)

• Full Gestalt-Popper protocol, all phases extended
• Auto-fix every finding P0/P1/P2
• Re-audit cycles until 100/100 (or 3 cycle cap)
• Output: forensic-grade with falsification proofs + telemetry
• Use case: pre-launch, security/compliance gate, "make it bulletproof"

Smart Selection Algorithm

When user says ambiguous request like "audit my project":

1. DETECT PROJECT TYPE
   - Check package.json: React/Next.js/Vue → UI audits relevant
   - Check requirements.txt/pyproject.toml: Python → no motion/uiux
   - Check .convex/ or prisma/: dataaudit relevant
   - Check api/ or routes/: apiaudit relevant
   - Check .github/workflows/: dxaudit + automationaudit
   - No src/ but docs/: feature/copy/seo only (docs project)

2. PARSE INTENT KEYWORDS (English + French)
   - "speed/fast/lent/lenteur" → perfaudit (+ seoaudit if web)
   - "security/sec/vuln/secure/sécurité" → secaudit + apiaudit
   - "design/visual/UI/UX/style" → uiuxaudit + motionaudit
   - "content/copy/messaging/text" → copyaudit
   - "accessibility/a11y/WCAG/handicap" → a11yaudit
   - "API/endpoint/contract" → apiaudit + dataaudit
   - "complete/missing/done/ship-ready" → featureaudit
   - "code/quality/refactor" → codeaudit + logicaudit
   - "retention/features/CPO/sticky" → retentionaudit
   - "data/schema/migration" → dataaudit
   - "automation/cron/scripts" → automationaudit
   - "bug/error/broken/runtime" → debugaudit
   - "redesign/refonte/dashboard" → refontaudit
   - "full/all/everything/complet" → ALL 23 audits

3. PICK POWER LEVEL
   - Default: Standard (Level 2)
   - User mentions "quick/fast/rapide" → Focused (Level 1, full phase depth)
   - User mentions "deep/forensic/production/launch/100" → Forensic (Level 3)

4. CHECK PROJECT MATURITY
   - Empty src/ or fresh scaffold → skip code-focused audits, run featureaudit+copyaudit
   - Mature codebase → all relevant
   - Pre-launch → add secaudit + a11yaudit + perfaudit (the "go-live trio")

Execution Plan Output

Before dispatching, OUTPUT a plan like:

🎯 AUDIT PLAN — {project_name}

Detected:
  Stack:     Next.js + Tailwind + Convex
  Maturity:  Production (12 months)
  Intent:    "make sure it's secure before launch"

Recommended (Power Level: Forensic):
  1. /secaudit       (OWASP + payment surfaces — primary)
  2. /apiaudit       (auth matrix + rate limits — secondary)
  3. /dataaudit      (RGPD + orphan records — context for /apiaudit)
  4. /a11yaudit      (legal compliance — go-live blocker)
  5. /perfaudit      (CWV — go-live blocker)

Estimated duration: 4-6h (parallel waves)
Estimated tokens: ~800K

Plan confirmation — session-scoped (Law L3 autonomy)

The confirmation step depends on WHO is running this:

• Interactive session (Home / a human is watching the terminal): print Approve? [y/n/customize] and WAIT for the answer before dispatching.

• Dispatched session (spawned by an oracle / worker / AISB — any tmux name matching oracle-*, *-linear, *-fix-*, *-dev-*, AISB-*, or started via dispatch-to-session.sh): do NOT ask. Pausing for [y/n] strands the run because nobody is watching (Law L3). Instead, LOG the decision and proceed at the default power level:

  Plan confirmation auto-approved (dispatched session — L3 autonomy).
  Power level: {detected default — Standard unless a forensic/go-live keyword was given}.
  Proceeding to dispatch.
  

  If the detected scope is genuinely ambiguous with no safe default, write the block-file (~/.omega/state/worker-blocked-<session>.json) with the candidate plan as the fallback and execute that fallback — never idle at a prompt.

Full Audit Mode

When user says "full audit" / "audit complet" / "tous les audits":

1. Dispatch ALL 23 audits in 3 parallel waves (file-safety partitioned):
   • Wave 1 (independent inventory, can parallel): codeaudit, logicaudit, dataaudit, apiaudit, seoaudit, featureaudit, retentionaudit, copyaudit, dxaudit, i18naudit, depaudit, privacyaudit
   • Wave 2 (consumes Wave 1 evidence): secaudit, perfaudit, debugaudit, automationaudit, releaseaudit, observabilityaudit
   • Wave 3 (experience bundle, after Wave 1): uiuxaudit, refontaudit, motionaudit, a11yaudit, flowaudit
2. After all done, generate audits/SYNTHESIS.md aggregating scores
3. Score the project: average /100 across all audits + flag any < 80
4. Telegram report with verdict + button to view each detailed report

State Tracking

Read audits/SYNTHESIS.md at start to know what's already done:

last_full_audit: 2026-05-13T12:00:00Z
scores:
  codeaudit: 92/A
  secaudit: 88/A
  uiuxaudit: 91/S
  ...
status:
  fresh:    [codeaudit, secaudit]  # < 7 days old
  stale:    [perfaudit]            # 7-30 days old
  expired:  [a11yaudit]            # > 30 days, recommend re-run

Output Convention

ALL audits MUST write to audits/.<audit-id>/, where <audit-id> is the exact identifier from skills/audits/registry.toml. Never write a machine audit root at the project root. The audit-orchestrator

• audit-tracker skills assume this canonical path.

Anti-patterns

• ❌ Running /codeaudit when project has no source code (use /dxaudit instead)
• ❌ Running /motionaudit on CLI/library project (it ABORTS automatically)
• ❌ Forensic level on every audit (token waste; use Standard unless go-live)
• ❌ Skipping the plan output (always print the plan). In an INTERACTIVE session also wait for [y/n/customize]; in a DISPATCHED session log auto-approval and proceed at default power (Law L3) — never idle at a [y/n] prompt nobody is watching
• ❌ Running audits in serial when waves allow parallelism
• ❌ Treating retentionaudit as fix-mode (it's READ-ONLY by design)

Workflow

User: "/audit-orchestrator security"
  ↓
You: parse "security" → secaudit + apiaudit + dataaudit
You: detect project at Standard level (no "deep/forensic" keyword)
You: emit plan markdown
     → interactive session: ask `[y/n/customize]` and wait
     → dispatched session: log auto-approval, proceed at default power (Law L3)
  ↓
User: "y"   (interactive only; dispatched skips straight to dispatch)
  ↓
You: dispatch 3 audits in parallel via tmux work sessions
You: monitor verdict.json files appearing under audits/.<audit-id>/
You: when all 3 done, write audits/SYNTHESIS.md
You: send Telegram report with aggregate score + per-audit links

When to invoke alternative skills

• For a SINGLE specific audit → user types /codeaudit directly (not via orchestrator)
• For audit setup / .gitignore / progress dashboard → use /audit-tracker
• For oracle dispatch of audit chain → use /aisb full

Dynamic-Workflow Orchestration (v2)

This section UPGRADES how the orchestrator runs — it does NOT change WHAT it selects, the 3 power levels, the wave partitioning, the audits/SYNTHESIS.md contract, or the aggregate /100 verdict. Same identity, same Gestalt-Popper doctrine (hinge-first selection, Popper falsification, runtime > code > comments). It just turns this conductor from a linear "decide → dispatch → wait → synthesize" loop into a fan-out → adversarially-verify → synthesize → loop-until-dry workflow. The orchestrator owns no scoring matrix of its own; its "verdict" is the aggregate it writes to SYNTHESIS.md, and that is untouched here.

When this orchestrator runs (any invocation — interactive, full, a level, or a keyword bundle), execute the steps below via the Workflow tool instead of grinding them in series.

1. Decompose into INDEPENDENT parallel tracks (fan-out)

The orchestrator's pre-dispatch work splits into file-disjoint, side-effect-free tracks that have no data dependency on each other. Run them concurrently in a single Workflow fan-out, not one after another:

• Track A — Stack/signal detection (§"Smart Selection Algorithm" step 1 + preamble §16): read package.json, lockfiles, .convex/, prisma/, api/, .github/workflows/, tailwind.config.*, env signals → emit project_signals_detected.
• Track B — Intent parsing (step 2): map the user's EN+FR keywords to the candidate audit set + chosen power level.
• Track C — Maturity probe (step 4): empty-scaffold vs mature vs pre-launch ("go-live trio") classification from git age + src density.
• Track D — Prior-state read: load existing audits/SYNTHESIS.md + audits/.<audit-id>/verdict.json freshness (fresh / stale / expired) so already-fresh audits are skipped, not re-run.

These tracks are read-only and touch different inputs → safe to parallelize (R-SCOPE: no shared writer). Join their outputs to compute the recommended audit list + level. The dispatch waves themselves stay exactly as defined in "Full Audit Mode" (Wave 1 read-only fan-out → Wave 2 consumers like secaudit that read apiaudit's verdict → Wave 3 UI bundle); each wave is itself a fan-out, and wave ordering is preserved because Wave 2/3 have real data dependencies on Wave 1 verdicts. Parallelize within a wave; serialize across waves.

2. ADVERSARIALLY VERIFY each finding before it reaches SYNTHESIS (>=2-of-3)

The orchestrator does not run forensic phases itself — its "findings" are the cross-audit signals it aggregates: each child audit's top findings, every cross_audit_confirmations elevation, and every score it is about to average into SYNTHESIS.md. Before ANY of these is accepted into the synthesis, subject it to three independent lenses and require >=2 to agree:

• Lens 1 — Reproduce: re-read the child audit's own verdict.json evidence chain (file:line → evidence → blast radius). Does the cited artifact actually exist and say what the finding claims? (Runtime > code > comments.)
• Lens 2 — Refute (Popper): actively try to kill it. Is the score an ABORT misread as a pass (a 401/403/empty-surface scored as green = ABORT, never a PASS — Law L5)? Did the audit hit its 5-iteration cap and leave needs_review? Is the "finding" an opinion with no falsifiable test?
• Lens 3 — Cross-check: does an independent audit corroborate it? A finding confirmed on the same file:line by a second audit is the existing elevation-to-CRITICAL mechanism (preamble §6); a contradiction between two audits is itself a finding to surface, not silently averaged away.

A signal that survives <2 lenses is killed — dropped from SYNTHESIS.md and recorded under a rejected_signals note (with the failing lens) so the kill is auditable (R-CITE). A delegate audit's own "done" is an input, never the verdict (R-VERIFY). Synthesis is the orchestrator's own job — never paste a child audit's summary verbatim as the aggregate truth (R-ORCH).

3. Synthesize survivors into the EXISTING aggregate (unchanged)

Fold only the surviving signals back into the orchestrator's existing outputs — the audits/SYNTHESIS.md aggregate, the average /100 across all audits + flag any < 80 scoring, and the plan/Telegram report described above. The synthesis format, the averaging rule, the grade bands, and the per-audit links are unchanged. This section adds a verification stage in front of synthesis; it does not alter the synthesis itself.

4. Loop-until-dry for unknown-size discovery

Audit selection is open-ended: a signal detected in Track A can pull in audits the keyword bundle never named (e.g. discovering stripe → /flowaudit --focus=payment; discovering prisma/ → /dataaudit + /apiaudit; discovering framer-motion → /motionaudit). Run selection as a convergence loop:

candidates = fan-out(Track A..D)         # initial set + level
do:
    run the next eligible wave (fan-out within the wave)
    read the new verdicts → adversarially verify (step 2)
    rescan project_signals_detected from fresh discovery artifacts
    add any newly-relevant audits to the pending waves
until: no new audit surfaces AND no wave has eligible un-run audits   # dry
then: synthesize (step 3)

Bound the loop the same way the family already bounds work: honor the 4h concurrency lock per audit, the 5-iteration fix cap inside each child audit, and the mission token budget (R-BUDGET — escalate near the cap, never silently overrun). "Dry" = a full pass adds zero new audits and leaves zero eligible un-run audits; only then is the aggregate verdict final.

Net effect

Same audits, same levels, same waves, same SYNTHESIS.md verdict — but selected by parallel discovery, hardened by >=2-of-3 adversarial verification so a single hallucinated child finding can't poison the aggregate, and exhaustive by loop-until-dry so a late-discovered stack signal never leaves a relevant audit unrun.

Sources

• 23 Quality Arsenal audits declared by skills/audits/registry.toml
• Helper docs: ARSENAL-ORCHESTRATION-PLAYBOOK.md, ARSENAL-INTERCONNECTIONS.md
• Public mirror: https://github.com/agentik-os/quality-arsenal