Batch Modules

Batch Module Creation

Automates the create → deploy-lab → build-module → test → tear-down-lab cycle across many gaps in parallel. Under the hood it invokes the batch-modules workflow (.claude/workflows/batch-modules.js), which pipelines items through four stages using a rolling pool of sub-agents. The plabs-lifecycle lock serializes terraform applies across concurrent items — no coordination code needed here beyond that.

This skill spawns multiple sub-agents concurrently. Only invoke when the user explicitly asks for batch/multi-agent module creation (e.g., "/batch-modules 5", "batch up 10", "work through the gap list"). Do not infer batching from a request to make a single module.

Prerequisites — check before invoking

The workflow's Preflight phase does the first check for you (SSO profile freshness). The others are still worth confirming up front so you don't fail 5 minutes in.

1. All plabs SSO profiles are alive: ./scripts/check-sso.sh plabs. Any expired profile will cause the workflow's Preflight phase to abort immediately. If it fails, run aws sso login --profile <name> for each expired one before invoking this skill. This is now a hard failure at workflow start — do not proceed and hope the sub-agents will handle it.
2. Attacker AWS identity is set and validated: ./pathrunner attacker validate. Some scenarios need the attacker exfil bucket or code artifacts bucket. If the attacker identity is a profile, also check that its SSO is fresh: ./scripts/check-sso.sh check <attacker-profile-name>.
3. plabs is initialized and pointing at a real AWS account: (cd ../pathfinding-labs && ./plabs status) runs without error. If not, tell the user to run ./plabs init in the labs repo first.
4. The lock is unlocked (or held only by a live process you know about): ./scripts/plabs-lifecycle.sh lock-info. If a stale/dead lock is shown, tell the user; don't force-unlock without asking.

Scenario starting credentials do not need to be pre-imported — the Enable stage auto-imports them via plabs-lifecycle.sh enable, so sub-agents in the Verify stage can just ./pathrunner identity switch <scenario-id>.

If any of these fail, stop and surface the problem — don't proceed and leak half-deployed labs.

Argument parsing

The user's $ARGUMENTS may contain:

• A count (integer, e.g. 5 or 10), or empty (default 5), or the word all (process every gap the list-gaps skill finds).
• --iterate N — Verify-stage fix-and-retry budget per module. Default 5. Set 1 to disable iteration and fail on first error; higher values give sub-agents more slack to chase down harder bugs.
• --verify-only — Skip the Build stage entirely. Modules must already exist. The workflow still enables labs, runs verify (with fix iteration), and disables. Useful for re-testing existing modules after changes or running regression checks.
• Explicit module IDs (e.g., iam-005 iam-007 bedrock-001) — When --verify-only is set, you can pass module IDs directly instead of discovering gaps. Build the gaps array from the IDs using {scenarioId}-to-admin as the default scenario.

Examples:

• /batch-modules 5 → 5 gaps, iteration budget 5
• /batch-modules 10 --iterate 3 → 10 gaps, iteration budget 3 per module
• /batch-modules all --iterate 1 → every gap, no iteration (fail fast)
• /batch-modules --verify-only iam-005 iam-007 → verify existing modules iam-005 and iam-007
• /batch-modules --verify-only all → verify all existing modules that have labs

Steps

Step 1: Discover targets

Normal mode (creating new modules): Invoke the list-gaps skill to produce the coverage-gap table. The output is a markdown table with columns: Path ID | Scenario | Module Exists | Category | Services.

Parse the table into a JSON array of { pathId, scenarioId, category, services } records. Filter to rows where Module Exists = No. If a service filter argument was provided, restrict further.

IMPORTANT: The scenarioId must be the plabs-registered ID (e.g., ecs-001-to-admin), NOT the directory name (e.g., ecs-001-iam-passrole+ecs-createcluster+...). The Scenario column from list-gaps already uses the correct format: {pathfinding-cloud-id}-{goal} where goal is typically to-admin.

If the resulting list is empty, tell the user "no gaps remaining" and stop.

Verify-only mode (--verify-only): If the user passed explicit module IDs (e.g., --verify-only iam-005 iam-007), build the gaps array directly from those IDs:

{ pathId: "iam-005", scenarioId: "iam-005-to-admin", category: "", services: "" }

If the user passed --verify-only all, invoke list-gaps and filter to rows where Module Exists = Yes (opposite of normal mode). If the user passed --verify-only N (a count), take the first N modules from the status manifest that have existing modules.

Step 2: Trim to the requested count

Take the first N entries from the gap list, where N is the parsed count (default 5, or all).

Show the user the exact list you're about to work on, one line per gap: pathId — scenarioId — category — services. Ask for confirmation before proceeding unless the user's original message clearly authorized the specific batch (e.g., "batch these five: X, Y, Z, ..."). This is a heavyweight operation — the deploy phase alone can take minutes and costs money.

Step 3: Bootstrap payload service directories

Skip this step in verify-only mode — modules already exist.

Multiple sub-agents will work on modules concurrently. If two agents both try to create pkg/payloads/{service}/ and add its blank import to main.go at the same time, they'll collide. Pre-create any missing service directories before the workflow starts.

Extract the unique services from the gap list (the primary service from each pathId, e.g., ecs from ecs-001). Run:

./scripts/ensure-payload-services.sh <service1> <service2> ...

This is idempotent — it skips services that already have a pkg/payloads/{service}/ directory and import. It also runs make build if anything changed, so the binary is ready for the sub-agents.

Step 4: Invoke the workflow

Call the Workflow tool with name: "batch-modules" and args set to the structured gap list plus the iteration budget. Add verifyOnly: true when in verify-only mode:

Workflow({
  name: "batch-modules",
  args: {
    gaps: [
      { pathId: "ecs-001", scenarioId: "ecs-001-to-admin", category: "new-passrole", services: "iam,ecs" },
      { pathId: "sqs-001", scenarioId: "sqs-001-to-admin", category: "new-passrole", services: "iam,sqs" },
      ...
    ],
    iterationBudget: 5,  // omit or pass 5 for default; 1 = single-shot, no fix loop
    verifyOnly: true,    // omit for normal mode; true skips Build stage
  }
})

The workflow runs a single up-front Preflight phase (SSO profile freshness), then pipelines each gap through Enable → Build → Verify → Disable. In verify-only mode, Build is a no-op passthrough. Sub-agents run concurrently, capped by the workflow runtime (min(16, cpu-cores-2)). The plabs-lifecycle script's file lock ensures at most one plabs apply is in flight at any moment; other items block on lock acquisition and continue as it releases. The Enable stage auto-imports each scenario's starting credentials into pathrunner's identity store, so Verify sub-agents just switch to <scenario-id> and start testing.

The Verify stage runs up to iterationBudget attempts per module. Each attempt: run go tests + test-module.sh full <id>, and if either fails, classify the failure. Pathrunner-side bugs (compile errors, wrong SDK calls, wrong ARN parsing, missing env vars in payloads, timing constants, etc.) get a minimal Edit-based fix; lab-side or environmental failures (missing scenario, SSO expired, drift) trigger a hard-stop with no code changes. Sub-agents never touch ../pathfinding-labs/** or ../pathfinding.cloud/**, never git-commit, never call plabs enable/disable. If the budget is exhausted without passing, the sub-agent returns success: false with the full fix history — Disable still runs, so labs aren't leaked.

Watch /workflows while it's running to see live progress.

Step 5: Report results

The workflow returns:

{
  summary: {
    attempted, enabled, built, tested, disabled, fullySuccessful,
    leakedLabs: [...],
    iterationBudget,
    iterations: { total, passedFirstTry, passedAfterFix, averageIterations },
    hardStopped: [{ pathId, reason }, ...]
  },
  results: [{ gap, enabled, built, tested, disabled, iterationsUsed, fixesApplied, verifyInfo, errors }, ...]
}

Summarize for the user:

• Fully successful (tested && disabled): list pathIds. Annotate each with (passed first try) or (passed after N fixes) from iterationsUsed. These are ready to commit.
• Passed after fixes: for any module where fixesApplied is non-empty, list the fix summaries so the user knows what changed before reviewing the diff. Especially important if the fixes touched shared code (pkg/discovery/, pkg/modules/).
• Built but failed verification (budget exhausted, no hard-stop): list pathIds, the final error, and the full fixesApplied history. These need human investigation — the sub-agent tried N times and couldn't crack it.
• Hard-stopped (summary.hardStopped): failures the sub-agent classified as lab/environmental. No code changes attempted. List pathId (reason) so the user knows which need lab-side or SSO fixes before re-running.
• Failed to build: list pathIds and their errors. Likely path YAML issues, unusual patterns, or compilation errors the sub-agent couldn't resolve.
• Leaked labs: summary.leakedLabs — labs that were enabled but disable failed. Tell the user to run ./scripts/plabs-lifecycle.sh disable <id> manually for each.
• Iteration cost signal: if iterations.total is high relative to modules passed, or averageIterations is >3, note it — that's a signal the sub-agents are grinding and either the budget can be tightened or the modules genuinely needed a lot of fixup.

Do NOT auto-commit the created modules. The user reviews the diff, then commits or asks for cleanup.

When to stop and ask

• Preflight SSO fails: the workflow's Preflight phase returned alive: false with an expiredProfiles list. Do NOT retry the workflow — tell the user which profiles need aws sso login, wait for them to confirm, then re-invoke.
• Stale lock at start (lock-info shows a lock older than the lease with a dead PID and reclamation didn't fire): don't force-unlock; ask the user.
• plabs status shows unexpected pre-deployed scenarios that overlap with the target list: ask whether to keep them or disable-and-redeploy. The workflow's enable stage handles alreadyDeployed=true gracefully, but the user should know.
• The gap list is >20 items and the user said "all": confirm the scope — that's a lot of AWS resources deploying serially through the lock.

Failure modes

• Workflow aborts at Preflight with aborted: 'sso-preflight': preflight.expiredProfiles lists which SSO profiles need refresh. Nothing was deployed; no cleanup needed. Refresh and re-invoke.
• Sub-agent Enable stage returns success: false with an SSO-expired error: rare — happens if a token expires mid-batch after Preflight passed. Prompt the user to refresh, then re-invoke for the remaining gaps.
• Workflow returns with leakedLabs non-empty: the enable succeeded but disable failed for those IDs. Their AWS resources are still deployed and billing. Run ./scripts/plabs-lifecycle.sh disable <id> manually, or ./scripts/plabs-lifecycle.sh force-unlock first if the lock is stuck.
• Workflow itself throws / is killed: any lab that was mid-enable when the workflow died is likely still deployed. Check plabs status and disable manually.
• A sub-agent silently returns unsuccessful with no error text: check /workflows for the sub-agent transcript. Common causes: sub-agent context ran out, tool call was denied by permission, source YAML file missing.
• Terraform state lock error inside a sub-agent's apply: the plabs-lifecycle lock should prevent this, but if it happens (e.g., the labs repo was applied directly outside this workflow), the sub-agent will report it via its structured output.
• Auto-import of scenario creds fails but apply succeeded: rare — usually means jq isn't installed or the pathrunner binary path is wrong. The lifecycle script logs a warning and continues; downstream sub-agents will fail their identity switch step. Run ./scripts/import-lab-creds.sh <scenario-id> manually to diagnose.

Related skills / scripts

• /list-gaps — produces the gap table this skill consumes
• /plabs-lifecycle — the lock-protected enable/disable/swap primitives sub-agents call
• /create-module — the module-creation flow each sub-agent follows in its Build stage
• /test-module — the live-lab test harness each sub-agent invokes in its Verify stage
• /cleanup-scenario — for manual cleanup if the workflow leaks labs