Validate Feature

Validate Feature

Deploy the feature locally with DEBUG logging, run security scans and behavioral tests against live services, check CI/CD status, and verify OpenSpec spec compliance. Produces a structured validation report and posts it to the PR.

Arguments

$ARGUMENTS - OpenSpec change-id (required), optionally followed by flags:

• --skip-e2e or --skip-playwright — skip the Playwright E2E phase
• --skip-ci — skip the CI/CD status check
• --skip-security — skip the Security Scan phase
• --phase <name>[,<name>] — run only specified phases (e.g., --phase smoke,security)
• --ephemeral — run write-capable validation steps in a disposable detached worktree
• --include-dirty — with --ephemeral, materialize staged, unstaged, and untracked source state instead of refusing a dirty checkout

Valid phase names: deploy, smoke, gen-eval, security, e2e, architecture, spec, logs, ci

Prerequisites

• Feature branch exists with implementation commits (default openspec/<change-id>, or the operator-mandated branch when OPENSPEC_BRANCH_OVERRIDE is set)
• A usable container runtime (docker or podman, daemon answering info) for the Deploy phase. Detection is the same as DockerStackEnvironment._detect_runtime: PATH presence is not enough.
• Approved OpenSpec proposal exists at openspec/changes/<change-id>/
• Run /implement-feature first if no implementation exists

Provider-Neutral Dispatch

When validation delegates checks or evidence review, treat the provider-neutral dispatch adapter as the canonical cross-provider path. Claude Code, Codex, Antigravity, Grok, and Pi are first-class providers when configured; Claude-specific harness examples are adapter internals, with inline validation as the fallback.

OpenSpec Execution Preference

Use OpenSpec-generated runtime assets first, then CLI fallback:

• Claude: .claude/commands/opsx/*.md or .claude/skills/openspec-*/SKILL.md
• Codex: .codex/skills/openspec-*/SKILL.md
• Fallback: direct openspec CLI commands

Coordinator Integration (Optional)

Use docs/coordination-detection-template.md as the shared detection preamble.

• Detect transport and capability flags at skill start
• Execute hooks only when the matching CAN_* flag is true
• If coordinator is unavailable, continue with standalone behavior

Local CLI Mutation Boundary

Validation writes reports, evidence, logs, and sometimes follow-up artifacts. In local CLI execution, validation MUST run inside the feature worktree or another managed worktree before the first write:

eval "$(python3 "<skill-base-dir>/../worktree/scripts/worktree.py" setup "$CHANGE_ID")"
cd "$WORKTREE_PATH"
python3 "<skill-base-dir>/../shared/checkout_policy.py" require-mutation

Read-only CI status checks may inspect from the shared checkout, but any report or evidence file must be written in a worktree so it lands on the PR branch. --ephemeral adds a disposable validation checkout inside this managed feature worktree boundary; it never makes the shared checkout writable.

Steps

0. Detect Coordinator and Recall Memory

At skill start, run the coordination detection preamble and set:

• COORDINATOR_AVAILABLE
• COORDINATION_TRANSPORT (mcp|http|none)
• CAN_LOCK, CAN_QUEUE_WORK, CAN_HANDOFF, CAN_MEMORY, CAN_GUARDRAILS

If CAN_MEMORY=true, recall relevant validation history:

• MCP path: recall
• HTTP path: "<skill-base-dir>/../coordination-bridge/scripts/coordination_bridge.py" try_recall(...)

On recall failure/unavailability, continue with validation and log informationally.

1. Determine Change ID and Configuration

# Parse change-id from argument or current branch
BRANCH=$(git branch --show-current)
CHANGE_ID=${ARGUMENTS%% --*}  # Everything before first flag
CHANGE_ID=${CHANGE_ID:-$(echo $BRANCH | sed 's/^openspec\///')}

# Detect worktree context and resolve OpenSpec path
# Note: detect auto-discovers context from the working directory;
# agent-id information is available via the worktree registry if needed.
eval "$(python3 "<skill-base-dir>/../worktree/scripts/worktree.py" detect)"
PROJECT_ROOT="${MAIN_REPO:-$(git rev-parse --show-toplevel)}"

Parse flags from $ARGUMENTS:

• --skip-e2e or --skip-playwright → set SKIP_E2E=true
• --skip-ci → set SKIP_CI=true
• --skip-security → set SKIP_SECURITY=true
• --phase <names> → set PHASES to comma-separated list; only run those phases
• --ephemeral → set EPHEMERAL=true
• --include-dirty → set INCLUDE_DIRTY=true; reject it unless EPHEMERAL=true

If --phase is provided, only the listed phases execute. If --phase includes phases other than deploy, assume services are already running (skip deploy and teardown).

2. Verify Prerequisites

# Resolve the expected feature branch — honors registry + OPENSPEC_BRANCH_OVERRIDE
eval "$(python3 "<skill-base-dir>/../worktree/scripts/worktree.py" resolve-branch "$CHANGE_ID")"
FEATURE_BRANCH="$BRANCH"

# Verify on feature branch
CURRENT_BRANCH="$(git branch --show-current)"
if [[ "$CURRENT_BRANCH" != "$FEATURE_BRANCH" ]]; then
  echo "ERROR: on '$CURRENT_BRANCH' but expected '$FEATURE_BRANCH' (source: $BRANCH_SOURCE)" >&2
  exit 1
fi

# Verify proposal exists
openspec show $CHANGE_ID

# Verify implementation commits exist
COMMIT_COUNT=$(git log --oneline main..HEAD | wc -l)
if [ "$COMMIT_COUNT" -eq 0 ]; then
  echo "ERROR: No implementation commits found on this branch."
  echo "Run /implement-feature $CHANGE_ID first."
  exit 1
fi

# Classify deployable surface (issue #432). Container-dependent phases are
# required only when this change has a running service; a skills/docs/openspec
# change records those phases as not applicable rather than skipped.
CHANGE_DIR="$(git rev-parse --show-toplevel)/openspec/changes/$CHANGE_ID"
SURFACE_JSON=$(python3 "<skill-base-dir>/scripts/gate_logic.py" --describe-surface \
  --change-dir "$CHANGE_DIR")
echo "$SURFACE_JSON"
DEPLOYABLE=$(printf '%s' "$SURFACE_JSON" | python3 -c \
  'import json,sys; print("true" if json.load(sys.stdin)["deployable"] else "false")')

# Check for a usable container runtime only when Deploy will run.
# Same predicate as DockerStackEnvironment._detect_runtime: PATH presence is
# not enough — `docker info` / `podman info` must succeed. Prefer docker when
# both work; fall through to podman when docker is installed but its daemon
# is down (issue #433).
if [ "$DEPLOYABLE" = "true" ]; then
  if RUNTIME=$(python3 "<skill-base-dir>/scripts/environments/docker_stack.py" --detect); then
    echo "Container runtime is available: $RUNTIME"
  else
    echo "ERROR: no usable container runtime (docker or podman)."
    echo "  A binary on PATH is not sufficient — its daemon must answer \`info\`."
    echo "  macOS: brew install --cask docker   OR   brew install podman"
    echo "  Linux: sudo systemctl start docker  OR   sudo systemctl start podman"
    exit 1
  fi
else
  echo "No deployable surface — Deploy/Smoke/Security/E2E are not applicable."
  echo "  Record those phases as **Status**: not applicable (not skipped)."
  DEPLOY_NOT_APPLICABLE=true
fi

If not on the feature branch, check out $FEATURE_BRANCH (which honors OPENSPEC_BRANCH_OVERRIDE). If no implementation commits exist, abort with guidance.

2.25. Enter Ephemeral Validation Scope (Optional)

When EPHEMERAL=true, enter the canonical prepare/finalize lifecycle before Step 2.5. The source is the current feature checkout, not the previously resolved PROJECT_ROOT (which may name the main repository from inside a managed worktree). This is an executable shell boundary: Steps 2.5 through 12 run from VALIDATION_PATH; Step 12.5 copies the durable allowlist back and removes the scratch checkout; Steps 13 and 14 then run from VALIDATION_SOURCE so PR comments, the session log, and its handoff are durable.

VALIDATION_HELPER="<skill-base-dir>/scripts/validation_worktree.py"
VALIDATION_STATE_FILE=$(mktemp "${TMPDIR:-/tmp}/validate-feature-state.XXXXXX")
INCLUDE_DIRTY_FLAG=""
[ "$INCLUDE_DIRTY" = "true" ] && INCLUDE_DIRTY_FLAG="--include-dirty"

if VALIDATION_PREPARE_OUTPUT=$(python3 "$VALIDATION_HELPER" prepare \
    --source "$PWD" \
    --change-id "$CHANGE_ID" \
    --state-file "$VALIDATION_STATE_FILE" \
    $INCLUDE_DIRTY_FLAG \
    ); then
  validation_json_field() {
    printf '%s' "$VALIDATION_PREPARE_OUTPUT" | \
      python3 -c 'import json,sys; print(json.load(sys.stdin)[sys.argv[1]], end="")' "$1"
  }
  if VALIDATION_SOURCE=$(validation_json_field source) && \
      VALIDATION_PATH=$(validation_json_field path) && \
      VALIDATION_VALIDATED_COMMIT=$(validation_json_field validated_commit) && \
      VALIDATION_VALIDATED_TREE=$(validation_json_field validated_tree); then
    export VALIDATION_SOURCE VALIDATION_PATH
    export VALIDATION_VALIDATED_COMMIT VALIDATION_VALIDATED_TREE
  else
    VALIDATION_PARSE_STATUS=$?
    python3 "$VALIDATION_HELPER" finalize --state-file "$VALIDATION_STATE_FILE" || true
    rm -f -- "$VALIDATION_STATE_FILE"
    echo "ERROR: could not parse ephemeral validation state" >&2
    exit "$VALIDATION_PARSE_STATUS"
  fi
else
  VALIDATION_PREPARE_STATUS=$?
  rm -f -- "$VALIDATION_STATE_FILE"
  echo "ERROR: could not prepare ephemeral validation" >&2
  exit "$VALIDATION_PREPARE_STATUS"
fi

finalize_ephemeral_validation() {
  validation_status="${1:-$?}"
  validation_cleanup_status=0
  trap - EXIT INT TERM
  cd "$VALIDATION_SOURCE" || validation_cleanup_status=$?
  python3 "$VALIDATION_HELPER" finalize --state-file "$VALIDATION_STATE_FILE" || \
    validation_cleanup_status=$?
  if [ "$validation_status" -eq 0 ]; then
    validation_status=$validation_cleanup_status
  fi
  return "$validation_status"
}
trap 'finalize_ephemeral_validation $?' EXIT
trap 'finalize_ephemeral_validation 130; exit 130' INT
trap 'finalize_ephemeral_validation 143; exit 143' TERM

cd "$VALIDATION_PATH"
PROJECT_ROOT="$VALIDATION_PATH"
OPENSPEC_PATH="$VALIDATION_PATH/openspec"

On a dirty checkout, omit --include-dirty to fail closed or pass it explicitly to reproduce the exact index, working-tree, and untracked state. The helper records VALIDATION_VALIDATED_COMMIT and VALIDATION_VALIDATED_TREE, copies only validation-report.md, validation-findings.json, and architecture-impact.md back to the feature checkout, and removes the scratch worktree even when an earlier step exits. Under a cloud harness whose environment profile already provides isolation, it logs a downgrade and runs in place.

2.5. Prepare Validation Artifacts

Preferred path:

• Use runtime-native verify/continue workflow (opsx:verify equivalent) for artifact guidance.

CLI fallback path:

openspec instructions validation-report --change "$CHANGE_ID"
openspec instructions architecture-impact --change "$CHANGE_ID"
openspec status --change "$CHANGE_ID"

Ensure validation-report.md and architecture-impact.md are updated in the change directory as part of this validation run.

3. Deploy Phase

Phase name: deploy Criticality: Critical when the change has a deployable surface; not applicable otherwise (issue #432). Do not record skipped for a phase that could never have applied.

if [ "$DEPLOY_NOT_APPLICABLE" = true ]; then
  echo "NOT APPLICABLE: no deployable surface — Deploy is not a skipped check."
  DEPLOY_RESULT="not applicable"
else
# Find docker-compose file
COMPOSE_FILE=$(find "$PROJECT_ROOT" -maxdepth 2 -name "docker-compose.yml" | head -1)

if [ -z "$COMPOSE_FILE" ]; then
  echo "SKIP: No docker-compose.yml found. Skipping Deploy phase."
  echo "  Smoke tests will run against already-running services."
  DEPLOY_SKIPPED=true
else
  COMPOSE_DIR=$(dirname "$COMPOSE_FILE")
  LOG_FILE="/tmp/validate-feature-${CHANGE_ID}-$(date +%s).log"

  echo "Starting services with DEBUG logging..."
  echo "  Compose file: $COMPOSE_FILE"
  echo "  Log file: $LOG_FILE"

  # Start services with DEBUG logging, redirect output to log file.
  #
  # When the compose file gates the API server behind a profile (e.g. the
  # agent-coordinator's `coordinator-api` service uses `profiles: [api]` so it
  # doesn't auto-start during simple `docker compose up`), pass
  # `COMPOSE_PROFILES` so the API process IS started here — without it the
  # smoke + e2e phases get connection-refused on the API port. Multiple
  # profiles can be comma-separated (`api,langfuse`).
  AGENT_COORDINATOR_DB_PORT=${AGENT_COORDINATOR_DB_PORT:-54322} \
  AGENT_COORDINATOR_REST_PORT=${AGENT_COORDINATOR_REST_PORT:-8081} \
  AGENT_COORDINATOR_REALTIME_PORT=${AGENT_COORDINATOR_REALTIME_PORT:-4000} \
  COMPOSE_PROFILES=${COMPOSE_PROFILES:-api} \
  LOG_LEVEL=DEBUG $RUNTIME compose -f "$COMPOSE_FILE" up -d --build 2>&1 | tee "$LOG_FILE"

  # Wait for health checks
  echo "Waiting for services to be healthy..."
  $RUNTIME compose -f "$COMPOSE_FILE" ps

  # Wait for PostgreSQL health check (up to 30 seconds)
  for i in $(seq 1 30); do
    if $RUNTIME compose -f "$COMPOSE_FILE" exec -T postgres pg_isready -U postgres > /dev/null 2>&1; then
      echo "PostgreSQL is ready"
      break
    fi
    sleep 1
  done

  # Wait for REST API health endpoint (up to 30 seconds — the API container
  # may need build time on first run + warmup before /health flips to 200)
  for i in $(seq 1 30); do
    if curl -sf http://localhost:${AGENT_COORDINATOR_REST_PORT:-8081}/health > /dev/null 2>&1; then
      echo "REST API is ready"
      break
    fi
    sleep 1
  done

  # Collect running container logs in background
  $RUNTIME compose -f "$COMPOSE_FILE" logs -f >> "$LOG_FILE" 2>&1 &
  LOG_PID=$!

  DEPLOY_RESULT="pass"
fi
fi

If Deploy fails, report the failure with Docker logs and skip to Teardown.

4. Smoke Phase

Phase name: smoke Criticality: Critical (stops validation on failure)

Run the reusable pytest smoke test suite against the live services. The suite is configurable via environment variables so it works with any deployed HTTP API.

if [ "$DEPLOY_NOT_APPLICABLE" = true ]; then
  echo "NOT APPLICABLE: no deployable surface — Smoke is not a skipped check."
  SMOKE_RESULT="not applicable"
else
# Configure for the target API (adjust per project)
export API_BASE_URL="${API_BASE_URL:-http://localhost:8000}"
export API_HEALTH_ENDPOINT="${API_HEALTH_ENDPOINT:-/health}"
export API_READY_ENDPOINT="${API_READY_ENDPOINT:-/ready}"
export API_AUTH_HEADER="${API_AUTH_HEADER:-X-Admin-Key}"
export API_AUTH_VALUE="${API_AUTH_VALUE:-$ADMIN_API_KEY}"
export API_PROTECTED_ENDPOINT="${API_PROTECTED_ENDPOINT:-/api/v1/settings/prompts}"
export API_CORS_ORIGIN="${API_CORS_ORIGIN:-http://localhost:5173}"

# Run smoke tests
SKILL_DIR="<skill-base-dir>"
pytest "$SKILL_DIR/scripts/smoke_tests/" -v --tb=short 2>&1
SMOKE_EXIT=$?

if [ $SMOKE_EXIT -eq 0 ]; then
  SMOKE_RESULT="pass"
elif [ $SMOKE_EXIT -eq 5 ]; then
  # Exit code 5 = no tests collected (services not running, all skipped)
  SMOKE_RESULT="skip"
  echo "SKIP: Services not running — smoke tests auto-skipped"
else
  SMOKE_RESULT="fail"
  SMOKE_FAILED=true
fi
fi

The smoke tests cover:

• Health: Health and readiness endpoints respond with 2xx
• Auth enforcement: No credentials → 401/403, valid credentials → 2xx, garbage credentials rejected
• CORS: Preflight returns correct Access-Control-* headers (skipped if CORS not configured)
• Error sanitization: Error responses don't leak filesystem paths, stack traces, internal IPs, or credentials
• Security headers: Content-Type set correctly, Server header not overly detailed, no X-Powered-By

If Smoke fails (SMOKE_EXIT != 0 and != 5), stop validation and skip to Teardown.

4b. Gen-Eval Phase (Optional)

Phase name: gen-eval Criticality: Non-critical (continues on failure)

Run generator-evaluator testing when interface descriptors exist for the project. This phase auto-detects descriptor files and selects between two modes:

• cli-augmented mode when both an interface descriptor AND an OpenSpec change directory at openspec/changes/<change-id>/specs/ exist. Gen-eval is invoked with --mode cli-augmented --openspec-change <change-id> so the generator seeds scenarios from the change's WHEN/THEN spec blocks.
• template-only mode (existing fallback) when descriptors exist but no OpenSpec change directory. Requires no CLI or SDK dependencies.
• Skipped when no descriptors are found.

# Auto-detect gen-eval descriptors
GENEVAL_DESCRIPTORS=$(find "$PROJECT_ROOT" -path "*/evaluation/descriptors/*.yaml" -type f 2>/dev/null)

if [ -z "$GENEVAL_DESCRIPTORS" ]; then
  echo "SKIP: No gen-eval descriptors found. Skipping gen-eval phase."
  GENEVAL_RESULT="skip"
else
  # Mode selection: cli-augmented requires both descriptor AND OpenSpec change dir
  GENEVAL_CHANGE_DIR="$PROJECT_ROOT/openspec/changes/$CHANGE_ID/specs"
  if [ -d "$GENEVAL_CHANGE_DIR" ]; then
    GENEVAL_MODE_FLAGS="--mode cli-augmented --openspec-change $CHANGE_ID"
    GENEVAL_MODE_LABEL="mode=cli-augmented"
    echo "gen-eval: $GENEVAL_MODE_LABEL (descriptor + OpenSpec change present at $GENEVAL_CHANGE_DIR)"
  else
    GENEVAL_MODE_FLAGS="--mode template-only --no-services"
    GENEVAL_MODE_LABEL="mode=template-only"
    echo "gen-eval: $GENEVAL_MODE_LABEL (no OpenSpec change at openspec/changes/$CHANGE_ID/specs/, falling back to template-only)"
  fi

  echo "Running gen-eval testing ($GENEVAL_MODE_LABEL)..."
  GENEVAL_FAILED=false

  for DESCRIPTOR in $GENEVAL_DESCRIPTORS; do
    echo "  Descriptor: $DESCRIPTOR"
    # Resolve the module root (parent of evaluation/) and cd into it.
    # Descriptor is at <project>/evaluation/descriptors/<name>.yaml — 3 levels up.
    GENEVAL_MODULE_ROOT=$(dirname "$(dirname "$(dirname "$DESCRIPTOR")")")
    GENEVAL_PYTHON="$GENEVAL_MODULE_ROOT/.venv/bin/python"
    if [ ! -f "$GENEVAL_PYTHON" ]; then GENEVAL_PYTHON="python3"; fi
    (cd "$GENEVAL_MODULE_ROOT" && GEN_EVAL_DATA_DIR="$GENEVAL_MODULE_ROOT/evaluation" "$GENEVAL_PYTHON" -m gen_eval \
      --descriptor "$DESCRIPTOR" \
      $GENEVAL_MODE_FLAGS \
      --report-format both \
      --output-dir "$PROJECT_ROOT/openspec/changes/$CHANGE_ID" 2>&1)
    GENEVAL_EXIT=$?

    if [ $GENEVAL_EXIT -ne 0 ]; then
      GENEVAL_FAILED=true
      echo "  gen-eval: FAIL for $DESCRIPTOR (exit $GENEVAL_EXIT)"
    else
      echo "  gen-eval: PASS for $DESCRIPTOR"
    fi
  done

  if [ "$GENEVAL_FAILED" = true ]; then
    GENEVAL_RESULT="fail"
    echo "Gen-eval: FAIL — One or more descriptors had failures (non-blocking)"
  else
    GENEVAL_RESULT="pass"
    echo "Gen-eval: PASS — All descriptors passed"
  fi
fi

Gen-eval failures are non-critical and do not block validation. Results are included in the validation report for informational purposes. cli-augmented mode failures (e.g., from prompt-injection attempts caught by the parser) still degrade gracefully — the validate-feature pipeline continues to subsequent phases.

5. Security Phase

Phase name: security Criticality: Non-critical (continues on failure)

Run security scanners (OWASP Dependency-Check and ZAP) against the live deployment using the existing security-review orchestrator.

if [ "$DEPLOY_NOT_APPLICABLE" = true ]; then
  echo "NOT APPLICABLE: no deployable surface — Security is not a skipped check."
  SECURITY_RESULT="not applicable"
elif [ "$SKIP_SECURITY" = true ]; then
  echo "SKIP: Security phase skipped (--skip-security flag)"
  SECURITY_RESULT="skip"
else
  echo "Running security scans against live deployment..."

  # Invoke the security-review orchestrator with the live API target
  python3 "<skill-base-dir>/../security-review/scripts/main.py" \
    --repo . \
    --out-dir docs/security-review \
    --zap-target "http://localhost:${AGENT_COORDINATOR_REST_PORT:-3000}" \
    --change "$CHANGE_ID" \
    --allow-degraded-pass 2>&1
  SECURITY_EXIT=$?

  if [ $SECURITY_EXIT -eq 0 ]; then
    # Exit 0 under --allow-degraded-pass can mean "no findings" OR "a scanner
    # never ran and we passed anyway". Those are different facts, so read the
    # gate reasons rather than trusting the exit code alone (D6).
    if grep -q "DEGRADED" docs/security-review/gate.json 2>/dev/null; then
      SECURITY_RESULT="DEGRADED"
      SECURITY_NOT_CHECKED=$(python3 -c "import json,sys; print('; '.join(r for r in json.load(open('docs/security-review/gate.json')).get('reasons', []) if 'DEGRADED' in r))" 2>/dev/null)
      echo "Security: DEGRADED — ${SECURITY_NOT_CHECKED:-a scanner did not run; coverage incomplete}"
    else
      SECURITY_RESULT="pass"
      echo "Security: PASS — No threshold findings"
    fi
  elif [ $SECURITY_EXIT -eq 10 ]; then
    SECURITY_RESULT="fail"
    echo "Security: FAIL — Threshold findings detected"
  elif [ $SECURITY_EXIT -eq 11 ]; then
    SECURITY_RESULT="DEGRADED"
    echo "Security: DEGRADED (INCONCLUSIVE) — Scanners could not run (check prerequisites)"
  else
    SECURITY_RESULT="fail"
    echo "Security: ERROR — Unexpected exit code $SECURITY_EXIT"
  fi
fi

The Security phase reuses the /security-review skill's scripts without requiring a separate invocation. The --allow-degraded-pass flag ensures missing prerequisites (Java, container runtime) degrade gracefully instead of blocking validation.

Write $SECURITY_RESULT into validation-report.md verbatim — including DEGRADED — together with a one-line "what was not checked and why". DEGRADED is not a pass: gate_logic.py blocks the pre-merge gate on a DEGRADED required phase unless the operator passes --accept-degraded Security, and that override is echoed into the gate summary:

# Blocks: Security could not be checked
python3 "<skill-base-dir>/scripts/gate_logic.py" openspec/changes/"$CHANGE_ID"/validation-report.md

# Proceeds, recording the override in the gate summary
python3 "<skill-base-dir>/scripts/gate_logic.py" openspec/changes/"$CHANGE_ID"/validation-report.md \
  --accept-degraded Security

6. E2E Phase

Phase name: e2e Criticality: Non-critical (continues on failure)

if [ "$DEPLOY_NOT_APPLICABLE" = true ]; then
  echo "NOT APPLICABLE: no deployable surface — E2E is not a skipped check."
  E2E_RESULT="not applicable"
elif [ "$SKIP_E2E" = true ]; then
  echo "SKIP: E2E phase skipped (--skip-e2e flag)"
  E2E_RESULT="skip"
else
  # Check if pytest-playwright is installed
  if python3 -c "import playwright" 2>/dev/null; then
    PLAYWRIGHT_AVAILABLE=true
  else
    PLAYWRIGHT_AVAILABLE=false
  fi

  # Check if E2E tests exist
  E2E_DIR=$(find "$PROJECT_ROOT" -path "*/tests/e2e" -type d | head -1)

  if [ -z "$E2E_DIR" ]; then
    echo "SKIP: No tests/e2e/ directory found. Skipping E2E phase."
    E2E_RESULT="skip"
  elif [ "$PLAYWRIGHT_AVAILABLE" = false ]; then
    echo "DEGRADED: E2E tests were NOT CHECKED because pytest-playwright is unavailable. To install:"
    echo "  pip install pytest-playwright"
    echo "  playwright install chromium"
    E2E_RESULT="DEGRADED"
  else
    echo "Running E2E tests from $E2E_DIR..."
    pytest "$E2E_DIR" -v --tb=short 2>&1
    E2E_EXIT=$?

    if [ $E2E_EXIT -eq 0 ]; then
      E2E_RESULT="pass"
    else
      E2E_RESULT="fail"
    fi
  fi
fi

6b. Architecture Diagnostics Phase

Phase name: architecture Criticality: Config-ratcheted via gates.architecture.mode in architecture.config.yaml (OpenSpec introduce-fitness-function-gates, D4)

The flip to blocking is a deliberate one-line config change made after clean_runs_before_flip (3) clean advisory runs, recorded with a date and rationale — not something a validation run decides for itself. Report the findings either way:

python3 -c "import sys; sys.path.insert(0, '<skill-base-dir>/scripts'); \
  import gate_logic; print(gate_logic.architecture_mode())"

Run the baseline architecture diff, flow validation, and structural linters against the changed files. The diff producer runs first so new dependency cycles exist as findings before gate_logic.architecture_status() evaluates the phase:

# Ensure architecture artifacts are current, immediately before the first read.
# `--ensure` is `--check` plus a staged refresh only when the check is not fresh,
# so on an already-fresh checkout it writes nothing. PYTHON must name the same
# interpreter this repository's architecture targets use: the check runs in-process
# and the pipeline runs in a subprocess, and if the two disagree about which
# optional grammars are importable they report permanent, unfixable drift.
ARCH_PY="${PYTHON:-python3}"
if "$ARCH_PY" "<skill-base-dir>/../refresh-architecture/scripts/run_architecture.py" --ensure --python "$ARCH_PY"; then
  ARCH_FRESHNESS="ensured"
else
  ARCH_FRESHNESS="DEGRADED"
  echo "DEGRADED: architecture artifacts could not be made current; the last known-good analysis is left intact but unverified. Report every architecture-derived finding below as unverified rather than as current." >&2
fi

# Get changed files relative to main
CHANGED_FILES=$(git diff --name-only main...HEAD | tr "\n" ",")

# --- Sub-phase 0: Baseline graph diff (new-cycle producer) ---
ARCH_BASE_SHA=$(git merge-base main HEAD)
ARCH_DIFF="docs/architecture-analysis/architecture.diff.json"
if [ -f Makefile ] && [ -f "docs/architecture-analysis/architecture.graph.json" ]; then
  echo "Running architecture baseline diff against $ARCH_BASE_SHA..."
  if make architecture-diff BASE_SHA="$ARCH_BASE_SHA"; then
    ARCH_NEW_CYCLES=$(python3 -c "import json; d=json.load(open(\"$ARCH_DIFF\")); print(d[\"summary\"][\"new_cycles\"])")
    ARCH_DIFF_RESULT=$(python3 -c "import json,sys; sys.path.insert(0, \"<skill-base-dir>/scripts\"); import gate_logic; d=json.load(open(\"$ARCH_DIFF\")); findings=[{\"category\": \"new_cycle\", \"description\": \"New dependency cycle: \" + \" -> \".join(c)} for c in d[\"details\"][\"new_cycles\"]]; print(gate_logic.architecture_status(findings))")
    echo "Architecture diff: $ARCH_NEW_CYCLES new cycle(s); gate status: $ARCH_DIFF_RESULT"
  else
    echo "DEGRADED: Architecture baseline diff was NOT CHECKED because the producer failed"
    ARCH_DIFF_RESULT="DEGRADED"
  fi
else
  echo "DEGRADED: Architecture baseline diff was NOT CHECKED because Makefile or graph artifacts are unavailable"
  ARCH_DIFF_RESULT="DEGRADED"
fi

# --- Sub-phase 1: Flow validation (validate_flows.py) ---
if [ -f "<skill-base-dir>/../validate-flows/scripts/validate_flows.py" ] && [ -f "docs/architecture-analysis/architecture.graph.json" ]; then
  echo "Running architecture flow validation on changed files..."
  # Scoped run: omit --output so the validator writes the scoped artifact
  # (architecture.diagnostics.scoped.json) rather than the committed full-scope
  # architecture.diagnostics.json, which only refresh-architecture full run
  # should produce.
  ARCH_DIAGNOSTICS="docs/architecture-analysis/architecture.diagnostics.scoped.json"
  python3 "<skill-base-dir>/../validate-flows/scripts/validate_flows.py" \
    --graph docs/architecture-analysis/architecture.graph.json \
    --files "$CHANGED_FILES" 2>&1
  ARCH_EXIT=$?

  if [ $ARCH_EXIT -eq 0 ]; then
    FLOW_RESULT="pass"
    ARCH_ERRORS=$(python3 -c "import json; d=json.load(open(\"$ARCH_DIAGNOSTICS\")); print(d[\"summary\"][\"errors\"])" 2>/dev/null || echo 0)
    ARCH_WARNINGS=$(python3 -c "import json; d=json.load(open(\"$ARCH_DIAGNOSTICS\")); print(d[\"summary\"][\"warnings\"])" 2>/dev/null || echo 0)
    if [ "$ARCH_ERRORS" -gt 0 ]; then
      FLOW_RESULT="fail"
    elif [ "$ARCH_WARNINGS" -gt 0 ]; then
      FLOW_RESULT="warn"
    fi
  else
    FLOW_RESULT="fail"
  fi
else
  echo "DEGRADED: Architecture flow validation was NOT CHECKED (missing scripts or artifacts)"
  echo "  ARCH_FRESHNESS=$ARCH_FRESHNESS — re-run the ensure call above to generate them"
  FLOW_RESULT="DEGRADED"
fi

# --- Sub-phase 2: Structural linters (dependency direction, file-size, naming) ---
echo "Running structural architecture linters..."
LINTER_OUTPUT=$(python3 "<skill-base-dir>/scripts/run_architecture_linters.py" \
  --files "$CHANGED_FILES" 2>&1)
LINTER_EXIT=$?

LINTER_FINDINGS=$(echo "$LINTER_OUTPUT" | head -1)  # JSON on stdout
if [ $LINTER_EXIT -eq 0 ]; then
  LINTER_RESULT="pass"
else
  LINTER_RESULT="fail"
fi

echo "Structural linters: $LINTER_RESULT"

# Aggregate without allowing a later passing sub-phase to erase an earlier
# failure or unavailable checker.
ARCH_RESULT="pass"
for SUBPHASE_RESULT in "$ARCH_FRESHNESS" "$ARCH_DIFF_RESULT" "$FLOW_RESULT" "$LINTER_RESULT"; do
  if [ "$SUBPHASE_RESULT" = "fail" ]; then
    ARCH_RESULT="fail"
  elif [ "$SUBPHASE_RESULT" = "DEGRADED" ] && [ "$ARCH_RESULT" != "fail" ]; then
    ARCH_RESULT="DEGRADED"
  elif [ "$SUBPHASE_RESULT" = "warn" ] && [ "$ARCH_RESULT" = "pass" ]; then
    ARCH_RESULT="warn"
  fi
done

Render architecture.diff.json in validation-report.md before running the hard gate, including its summary.new_cycles count and every details.new_cycles path. Also report broken flows, missing test coverage, orphaned code, disconnected endpoints, dependency direction violations, oversized files, and naming convention issues. Structural linter findings are output in review-findings.schema.json format for integration with the consensus synthesizer.

7. Spec Compliance Phase (via Change Context)

Phase name: spec Criticality: Non-critical (continues on failure) — EXCEPT the task-drift gate (7.0) and the requirement-traceability gate (7.0b), which are CRITICAL within this phase.

7.0. Task Checkbox Drift Gate (CRITICAL)

Before verifying requirements against the live system, enforce that tasks.md reflects commit reality. Drift between the two is a spec-compliance failure: the plan document either overstates completeness (dangerous) or understates it (bookkeeping debt that breaks archive-time invariants).

# Run from the change's worktree or feature branch
TASKS_FILE="openspec/changes/<change-id>/tasks.md"
UNCHECKED=$(grep -cE "^\s*- \[ \]" "$TASKS_FILE" 2>/dev/null || echo 0)

# Count commits on the feature branch since divergence from main
# (excludes commits on main that the branch inherited)
COMMIT_COUNT=$(git rev-list --count main..HEAD 2>/dev/null || echo 0)

if [ "$UNCHECKED" -gt 0 ] && [ "$COMMIT_COUNT" -gt 0 ]; then
  echo "FAIL: task checkbox drift detected"
  echo "  $TASKS_FILE has $UNCHECKED unchecked boxes"
  echo "  branch has $COMMIT_COUNT commit(s) since main"
  echo ""
  echo "Either:"
  echo "  (a) complete the remaining tasks — in which case this validation run is premature"
  echo "  (b) reconcile — flip checkboxes for tasks whose code has landed (new commit, do NOT amend)"
  echo "  (c) defer — move genuinely-skipped tasks to deferred-tasks.md"
  echo ""
  echo "Do not proceed to requirement verification with drifted tasks.md."
  exit 1  # CRITICAL failure — halts the spec phase
fi

Why this is CRITICAL: Archive validation (openspec archive) checks the tasks artifact's overall status, not individual checkboxes — meaning drift can slip through archive-time and leave inaccurate history. Catching it here, before the archive path, ensures the spec phase is the single source of truth for "does the plan document match what was built?" Per the incident log, specialized-workflow-agents shipped 29 tasks' worth of implementation to main with 0/29 checkboxes flipped because the validation gate didn't catch the drift (this check was added 2026-04-22 in response).

In CI-vs-local behavior: In local validation, exit 1 halts the phase immediately. In CI-invoked validation (where halting would abort merge-gate automation unhelpfully), record the drift as a CRITICAL finding in validation-report.md under "Phase Results" with Result=fail and Details listing the specific unchecked task IDs — do not silently continue.

7.0b. Requirement-to-Contract Traceability Gate (CRITICAL, change-scoped)

This is the enforcement point for the requirement-to-contract edge: a contracted operation the change touches must cite the requirements it serves, and a requirement the change adds must be cited or excluded, before the change validates. Pre-existing violations the change did not create are reported by the gate without failing it, so this wiring blocks only new debt — it never fails a change for a gap it did not introduce.

Detection first, exactly like the Gen-Eval phase (4b) above: skills/validate-feature/ ships via install.sh into consumer repositories that have neither packages/gen-eval/ nor openspec/contracts/, where the gate cannot run at all. Wiring it unconditionally would fail every validation in every downstream repo, so the SKIP is printed explicitly rather than the step being silently absent — an unprinted skip and a passing gate are the same observation in a log.

# This gate evaluates the tree UNDER VALIDATION, not the shared checkout.
# PROJECT_ROOT resolves to MAIN_REPO inside a managed worktree, and a gate
# rooted there would evaluate main's contracts instead of this branch's —
# SKIPping on every worktree validation and validating the wrong tree after
# merge. The gate therefore derives its own root from the current tree.
TRACE_ROOT="$(git rev-parse --show-toplevel)"
TRACE_GATE="$TRACE_ROOT/packages/gen-eval/scripts/check_traceability.py"
TRACE_CONTRACTS_DIR="$TRACE_ROOT/openspec/contracts"

if [ ! -f "$TRACE_GATE" ]; then
  echo "SKIP: requirement-traceability gate unavailable ($TRACE_GATE not found). Skipping."
  TRACE_RESULT="skip"
elif [ ! -d "$TRACE_CONTRACTS_DIR" ]; then
  echo "SKIP: requirement-traceability gate unavailable ($TRACE_CONTRACTS_DIR not found). Skipping."
  TRACE_RESULT="skip"
else
  TRACE_PYTHON="$TRACE_ROOT/packages/gen-eval/.venv/bin/python"
  if [ ! -f "$TRACE_PYTHON" ]; then TRACE_PYTHON="python3"; fi
  # Bare, never piped — a pipeline's $? is the last stage's exit status, so
  # `check_traceability.py | tail` would report tail's 0 on a failing gate.
  #
  # errexit is suspended across the capture. This fragment is pasted into
  # whatever shell the running agent has, and a failing gate under `set -e`
  # aborts on the assignment itself: the shell dies before `echo "$TRACE_OUTPUT"`
  # ever runs, so the violation text the report is supposed to quote is lost and
  # the operator sees a bare non-zero exit. The gate failing is the case this
  # phase exists to report, so it is precisely the case that must not kill the
  # reporter. Saved and restored rather than left off, so nothing after this
  # block silently loses errexit.
  case $- in *e*) _TRACE_HAD_ERREXIT=1;; *) _TRACE_HAD_ERREXIT=0;; esac
  set +e
  TRACE_OUTPUT=$(cd "$TRACE_ROOT/packages/gen-eval" && "$TRACE_PYTHON" scripts/check_traceability.py \
    --scope change --change "$CHANGE_ID")
  TRACE_EXIT=$?
  [ "$_TRACE_HAD_ERREXIT" = "1" ] && set -e
  echo "$TRACE_OUTPUT"
  if [ $TRACE_EXIT -ne 0 ]; then
    echo "FAIL: requirement-traceability gate exited $TRACE_EXIT"
    TRACE_RESULT="fail"
  else
    echo "PASS: requirement-traceability gate"
    TRACE_RESULT="pass"
  fi
fi

# Skip and pass both leave this sub-step at exit 0; fail propagates the
# gate's own non-zero status so a caller chaining this fragment observes it
# without re-deriving TRACE_RESULT.
[ "${TRACE_RESULT:-}" = "fail" ] && exit "$TRACE_EXIT"
exit 0

Note: --scope change --change "$CHANGE_ID" is the only invocation this skill makes. It shadows the archive with this change's own delta and reports touched violations only — pre-existing gaps the change did not create are reported, never failed (this is what makes the gate safe to make blocking). The full-capability sweep (every requirement, every capability, unbounded by change scope) is a separate, CI-only invocation and is never run here.

In CI-vs-local behavior, mirroring 7.0: in local validation, a fail result halts further spec-compliance work — do not proceed to 7.1's per-requirement matrix update as if the phase passed. In CI-invoked validation, record TRACE_RESULT=fail as a CRITICAL finding in validation-report.md under "Phase Results" with Result=fail and Details set to $TRACE_OUTPUT (it already names the violating operations and requirements) — do not silently continue past it.

A skip result is not a failure: record it as a skipped sub-step (○) and proceed normally.

7.1. Requirement Traceability (per-requirement live verification)

Use the change-context.md traceability matrix as the spec compliance artifact:

1. Read change-context.md from the change directory ($OPENSPEC_PATH/changes/<change-id>/change-context.md).

   • If it does not exist (pre-existing change implemented before this artifact was introduced), generate the skeleton now: read spec delta files from specs/, extract SHALL/MUST clauses, and create rows with Req ID, Spec Source, Description, and Test(s) derived from git diff --name-only main..HEAD.

2. For each row in the Requirement Traceability Matrix, verify the requirement against the live system:

   • API scenarios: Make HTTP requests to the running service and verify responses
   • MCP tool scenarios: Invoke MCP tools via the Python module and check results
   • Database scenarios: Query PostgreSQL directly and verify state
   • Configuration scenarios: Check file existence, content, or environment variables

3. Update the Evidence column for each row:

   • pass <short-SHA> — requirement verified successfully against the live system
   • fail <short-SHA> — requirement verification failed (include brief reason)
   • deferred <reason> — cannot verify in this environment (e.g., requires production)

4. Update Coverage Summary with final counts: requirements traced, tests mapped, evidence collected, gaps, and deferred items.

Report results sourced from the updated change-context.md:

Spec Compliance Results (from change-context.md):
  ✓ skill-workflow.1: Change context artifact generated during implementation
  ✓ skill-workflow.2: 3-phase incremental generation
  ✗ skill-workflow.3: TDD enforcement — test written after implementation
  ✓ skill-workflow.4: Validation report references change-context.md

7.5. Work Package Evidence Completeness [local-parallel+]

Phase name: evidence Criticality: Non-critical (continues on failure)

This phase runs only when work-packages.yaml exists at openspec/changes/<change-id>/. It audits that all work packages produced valid results and contract compliance evidence is present.

For each work package, validate its result (if artifacts/<package-id>/work-queue-result.json exists):

python3 "<skill-base-dir>/../validate-packages/scripts/validate_work_result.py" \
  artifacts/<package-id>/result.json

Checks per package:

• Result JSON validates against work-queue-result.schema.json
• contracts_revision and plan_revision match work-packages.yaml
• scope_check.passed is true
• verification.passed is true
• No unresolved escalations with disposition fix or escalate

Cross-package consistency:

• No two packages report modifications to the same file
• All packages used the same contracts_revision and plan_revision

If change-context.md exists, populate the Evidence column from work-queue results.

8. Log Analysis Phase

Phase name: logs Criticality: Non-critical (continues on failure)

Scan the collected log file for warning signs:

if [ -f "$LOG_FILE" ]; then
  echo "Analyzing logs: $LOG_FILE"
  ech

…(truncated)