Subagent Task QA

Subagent Task QA

QA and verify sub-agent task adapters, execution engines, and task DAGs by driving the REAL agent runner binary under strict sandbox isolation. Unit tests alone do not count as live QA: mock tests verify syntax, but live drivers provide deterministic proof of orchestration, child process isolation, and terminal state recovery.

Golden rules

• Evidence lives at exactly one path. Every artifact goes under ${EVIDENCE_DIR:-.evidence/subagent-qa}/<slug>/. Use the canonical path resolver: reject traversal (..), path separators, absolute paths, and stray roots.
• The real agent dir stays untouched. The live drivers build their own isolated ${TASK_AGENT_SANDBOX_DIR} and deliberately IGNORE user-global agent directories. Report the driver's changed-path fields and the isolated sandbox path.
• No binary means SKIP, not silence. When the target runner binary is absent, the live drivers report SKIP or FAIL in their final JSON rather than silently degrading to the real home directory. A SKIP is not a pass — document it explicitly.
• The captured JSON is the evidence. No evidence file on disk means the QA did not happen, blocking promotion, commit, and push.

Resolve the evidence directory first

# Define task agent binary (default: task-runner or senpi)
export TASK_AGENT_BIN="${TASK_AGENT_BIN:-task-runner}"
export EVIDENCE_DIR="${EVIDENCE_DIR:-.evidence/subagent-qa}"

ev="$(node scripts/resolve-evidence-dir.mjs \
  --repo-root "$(git rev-parse --show-toplevel)" --slug <YYYYMMDD>-<short-slug>)"
mkdir -p "$ev"

A slug must be ONE relative segment of lowercase letters, digits, and hyphens (e.g., 20260902-task-dag-contract). Separators, . / .., traversal, absolute paths, and non-git roots are rejected with exit code 1.

Router: pick your case

Writing the Evidence Report

Every run must generate $ev/README.md containing:

1. What was tested (exact scenario and flags).
2. What was observed (verifiable metrics, terminal states, child process PIDs).
3. Sandbox cleanup proof: assert all child task sandboxes and background processes are terminated.