Ralph Planning Audit

Ralph Planning Audit

Pre-execution planning audit for Ralph PRDs. Run this BEFORE Ralph starts executing to catch plan-level issues — wrong dependencies, missing stories, file conflicts, infeasible scope, contract violations — while they're cheap to fix.

When to Use

• After writing a prd.json but before running ralph.sh
• After significantly restructuring stories in an existing PRD
• When onboarding to a PRD someone else wrote

Trigger Phrases

• /ralph-planning-audit
• /plan-audit
• "audit the plan"
• "dry run the PRD"
• "review the ralph plan"

Input Detection

Parse user input for:

• PRD path: explicit path, or auto-detect by globbing **/ralph/prd/prd*.json
• Service/feature name: e.g., "backend-hardening", "tesla", "logo-v2"

If multiple PRDs found, list them and ask user to pick.

Phase 0: Build Context

Read these files (skip if missing):

1. AGENTS.md — canonical patterns, conventions, tech stack
2. CLAUDE.md — project constraints, dev commands
3. package.json — dependencies, scripts
4. The PRD file itself — all stories, deps, acceptance criteria
5. frontend-api-contract.md or equivalent — if stories touch API surfaces
6. Any doc referenced in the PRD's documentation field

Construct a {context block} containing:

• Tech stack, project structure, key conventions
• Dev commands and quality gates
• Full PRD contents (all stories with their fields)
• Dependency graph (derive from depends_on fields)
• API contract summary (if applicable)

Also extract:

• All files referenced across all stories (files_to_modify + files_to_create)
• File collision map — which stories touch the same files
• Story count, total priority range, dependency depth

Phase 1: Quick Validation (Sequential, No Agents)

Before spinning up agents, do fast structural checks yourself:

1A: PRD Schema Validation

For each story, verify:
- Has id, title, priority, status, description
- Has acceptance_criteria (non-empty array)
- Has files_to_modify or files_to_create (at least one non-empty)
- depends_on references only valid story IDs
- No circular dependencies
- Priority numbers are unique (warn if not)

1B: Dependency Graph Validation

- Build adjacency list from depends_on
- Topological sort — if cycle detected, report immediately
- Identify stories with no dependents (leaf nodes) and no dependencies (roots)
- Flag stories whose depends_on includes IDs not in the PRD

1C: File Collision Detection

- Build map: file_path -> [story IDs that touch it]
- Any file touched by 3+ stories is HIGH risk
- Any file touched by 2 stories that aren't in a dependency chain is MEDIUM risk
- Report the collision map

Report Phase 1 results to user:

Phase 1: Quick Validation
=========================
Stories: {count} ({incomplete} incomplete, {complete} complete)
Dependency graph: {VALID / CYCLES: list}
File collisions: {count} ({HIGH_count} high-risk, {MEDIUM_count} medium-risk)
Schema issues: {count or "none"}

If cycles found, stop and fix before proceeding.

Phase 2: Planning Audit Agents (Parallel)

Launch 5 agents simultaneously, all run_in_background: true, all subagent_type: "Explore".

CRITICAL: Every agent gets the full context block AND the Agent Roster.

Include in every agent prompt:

## Agent Roster (you are Agent {N})
1. Codebase Reality Agent — verifies files exist, code assumptions are correct
2. Sequencing & Dependency Agent — execution order, missing deps, parallel opportunities
3. Scope & Feasibility Agent — story sizing, acceptance criteria quality, one-context-window fit
4. Contract & Integration Agent — API contract compliance, cross-system effects
5. Gap Analysis Agent — missing stories, uncovered edge cases, incomplete coverage

If you find something outside your domain, note it as:
  CROSS-REF [Agent N]: [what they should check]

Agent 1: Codebase Reality Agent

Does the plan match what's actually in the code?

{context block}

{agent roster}

## Your Focus: Codebase Reality Check

For EVERY story in the PRD:

1. **File existence**: Do all `files_to_modify` actually exist? Read each one.
2. **Code assumptions**: Does the story description match what's actually in the file?
   - If a story says "fix line ~332 where customerEmail defaults to empty string" — verify that line exists and says what the story thinks it says.
   - If a story says "add column to schema" — verify the column doesn't already exist.
   - If a story says "update endpoint to accept new param" — verify the endpoint exists and doesn't already accept it.
3. **Import/dependency assumptions**: If a story assumes a module exists or exports something, verify it.
4. **Files NOT in the PRD**: Are there files that obviously need changes but aren't listed in any story? (e.g., if a schema changes, does a migration story exist?)

## OUTPUT FORMAT (MANDATORY)
CODEBASE REALITY REPORT
=======================
Stories Verified: {count}

Per-Story:
- {STORY_ID}: {PASS / ISSUES}
  {if issues: list each with file:line and what's wrong}

Missing from PRD (files that need changes but no story covers them):
- {file}: {why it needs changes}

Phantom References (files/lines that don't exist as described):
- {STORY_ID}: {what's wrong}

CROSS-REF:
- [Agent N]: [what they should verify]

Agent 2: Sequencing & Dependency Agent

Is the execution order correct?

{context block}

{agent roster}

## Your Focus: Sequencing & Dependencies

1. **Dependency correctness**: For each `depends_on`, verify the dependency is real:
   - Does story B actually need story A's output to work?
   - Or could B run independently?
   - Are there MISSING dependencies? (Story C modifies a file that story A creates, but C doesn't depend on A)

2. **Execution order optimization**:
   - Given the dependency graph, what's the critical path?
   - Which stories can run in parallel (same priority, no deps between them)?
   - Are there unnecessary sequential constraints slowing things down?

3. **File conflict sequencing**:
   - If stories A and B both modify `worker/src/index.ts`, is there a dependency between them?
   - If not, will they conflict? (Both changing the same function = conflict. Different functions = safe.)
   - Read the actual files to determine if parallel execution of same-file stories is safe.

4. **Priority ordering**:
   - Does priority order match dependency order? (A story shouldn't have higher priority than its dependency)
   - Are there priority ties that could cause ambiguity?

## OUTPUT FORMAT (MANDATORY)
SEQUENCING REPORT
================
Critical Path: {story IDs in order, with total depth}
Parallelizable Batches:
  Batch 1: [{story IDs}]
  Batch 2: [{story IDs}]
  ...

Dependency Issues:
- {STORY_ID}: {MISSING_DEP / UNNECESSARY_DEP / WRONG_ORDER}
  Reason: {explanation}

File Conflict Risks:
- {file}: touched by [{story IDs}]
  Safe to parallelize: {YES/NO}
  Reason: {different functions / same function / etc.}

Priority Corrections Needed:
- {STORY_ID}: priority {current} should be {suggested} because {reason}

CROSS-REF:
- [Agent N]: [what they should verify]

Agent 3: Scope & Feasibility Agent

Are stories right-sized and well-defined?

{context block}

{agent roster}

## Your Focus: Scope & Feasibility

For EVERY story:

1. **One-context-window fit**: Can this story be completed in a single Claude context window?
   - Count: files to read + files to modify + acceptance criteria complexity
   - If a story touches 5+ files or has 8+ acceptance criteria, it's likely too large
   - Flag stories that should be split

2. **Acceptance criteria quality**:
   - Are criteria specific and testable? ("pnpm typecheck passes" = good. "Works correctly" = bad.)
   - Do criteria cover all the changes described?
   - Are there acceptance criteria that contradict each other?
   - Is "pnpm typecheck passes" included? (Required per project conventions)

3. **Description sufficiency**:
   - Does the description give enough context for an autonomous agent to implement?
   - Are there ambiguous decisions the agent would need to make?
   - Would the agent need to ask clarifying questions?

4. **Risk assessment**:
   - Which stories have the highest risk of breaking something?
   - Which stories are most likely to need human intervention?
   - Flag any story that requires external access (API keys, other repos, manual steps)

## OUTPUT FORMAT (MANDATORY)
SCOPE & FEASIBILITY REPORT
==========================
Stories Assessed: {count}

Too Large (split recommended):
- {STORY_ID}: {reason} -> suggested split: [{sub-story descriptions}]

Weak Acceptance Criteria:
- {STORY_ID}: {what's missing or vague}

Ambiguous Descriptions:
- {STORY_ID}: {what decision is left unclear}

External Dependencies (may block Ralph):
- {STORY_ID}: requires {what} — {suggestion}

Risk Ranking (highest risk first):
1. {STORY_ID}: {risk description}
2. {STORY_ID}: {risk description}
3. ...

CROSS-REF:
- [Agent N]: [what they should verify]

Agent 4: Contract & Integration Agent

Will these changes maintain system contracts?

{context block}

{agent roster}

## Your Focus: Contract & Integration Compliance

1. **API contract preservation**:
   - Read the API contract doc (frontend-api-contract.md or equivalent)
   - For every story that modifies an API endpoint, verify the change preserves the contract
   - Flag any story that would break the frontend
   - Check: response shapes, status codes, field names (snake_case), error formats

2. **Cross-system effects**:
   - If a story changes the DB schema, do all downstream consumers handle the change?
   - If a story changes a Trigger.dev job's payload, does the trigger caller match?
   - If a story changes env vars, is .env.example updated?

3. **Backward compatibility**:
   - Are there existing customers/orders that would be affected?
   - Do migration stories handle existing data?
   - Is there a rollback path if something goes wrong?

4. **Frontend handoff alignment** (if frontend handoff doc exists):
   - Do the planned changes align with what the frontend team was told?
   - Will any story contradict the frontend handoff doc?
   - Are there backend changes the frontend expects that no story covers?

## OUTPUT FORMAT (MANDATORY)
CONTRACT & INTEGRATION REPORT
=============================
API Contract: {PRESERVED / AT RISK}

Per-Story Contract Impact:
- {STORY_ID}: {SAFE / RISK: description}

Cross-System Effects:
- {STORY_ID}: {effect on other systems}

Backward Compatibility:
- {STORY_ID}: {compatible / BREAKING: what breaks}

Frontend Handoff Alignment:
- {ALIGNED / GAPS: list what's missing or contradictory}

CROSS-REF:
- [Agent N]: [what they should verify]

Agent 5: Gap Analysis Agent

What's missing from the PRD?

{context block}

{agent roster}

## Your Focus: Gap Analysis

1. **Read every handoff doc, plan doc, and known-issues section** referenced in the project.
   - Compare against the PRD stories
   - Flag any known issue or planned feature that has NO corresponding story

2. **Edge cases not covered**:
   - If a story adds error handling, does it cover all error paths?
   - If a story changes a flow, are rollback/retry scenarios handled?
   - If a story adds a feature, is the feature's removal/deprecation path considered?

3. **Testing gaps**:
   - Are there stories for adding tests? (If test coverage is zero and this matters)
   - Do high-risk stories have verification steps beyond typecheck?

4. **Documentation gaps**:
   - Will the changes require AGENTS.md updates beyond what's in the PRD?
   - Are there README or setup guide updates needed?
   - Do new env vars need documentation?

5. **Operational gaps**:
   - Is there a story for monitoring/alerting on new features?
   - Is there a deployment/migration ordering concern?
   - Are there manual steps between stories that Ralph can't do?

## OUTPUT FORMAT (MANDATORY)
GAP ANALYSIS REPORT
==================
Known Issues Coverage: {X}/{Y} covered by stories

Missing Stories (should exist but don't):
- {title}: {why it's needed} — suggested priority: {N}, depends_on: [{IDs}]

Edge Cases Not Covered:
- {STORY_ID}: {edge case missing}

Testing Gaps:
- {description}

Documentation Gaps:
- {what needs updating}

Operational Concerns:
- {deployment order, manual steps, monitoring}

CROSS-REF:
- [Agent N]: [what they should verify]

Phase 2.5: Cross-Examination

After all 5 agents complete:

1. Collect all reports
2. Extract all CROSS-REF items
3. Identify overlapping findings (2+ agents flagged same story/issue)
4. Identify contradictions (agents disagree about a story)

For substantive cross-refs and contradictions, launch targeted verification agents:

subagent_type: "Explore"
run_in_background: true

Prompt:

{context block}

## Cross-Examination Task

Agent {N} ({domain}) found:
  {finding with context}

Agent {M} ({domain}) is asked to verify:
  {the cross-ref question}

Other agents said about the same area:
  {related findings}

Your job: Read the relevant files/stories and either CONFIRM, REFUTE, or EXPAND.

## OUTPUT FORMAT (MANDATORY)
CROSS-EXAM RESULT
=================
Original Finding: [{agent} {finding ID}]
Verdict: [CONFIRMED / REFUTED / EXPANDED]
Evidence: [what you found]
Recommendation: [what to change in the PRD]

Guidelines:

• Only cross-examine HIGH findings, contradictions, and explicit CROSS-REFs
• Batch related items into single agents
• Limit to ~3 cross-exam agents
• Skip if no substantive cross-refs

Phase 3: Consolidation

After all agents (including cross-exam) complete, build the consolidated report:

Ralph Planning Audit — {PRD Name}
===================================
PRD: {path}
Stories: {count} ({incomplete} to execute)
Date: {date}

Phase 1 (Structural):
  Schema: {VALID/ISSUES}
  Dependencies: {VALID/CYCLES}
  File Collisions: {count} ({risk breakdown})

Phase 2 (Deep Analysis):
  Codebase Reality: {X}/{Y} stories verified against actual code
  Sequencing: Critical path depth {N}, {M} parallelizable batches
  Feasibility: {X} stories need splitting, {Y} have weak criteria
  Contract: {PRESERVED/AT RISK}
  Gaps: {X} missing stories identified

Cross-Agent Confirmations (highest confidence):
  {findings confirmed by 2+ agents or cross-examination}

MUST-FIX BEFORE EXECUTION:
  1. {finding — what to change in the PRD}
  2. {finding — what to change in the PRD}

SHOULD-FIX (improves success rate):
  1. {finding — what to change in the PRD}

NICE-TO-HAVE:
  1. {finding}

Missing Stories to Add:
  {list from gap analysis, with suggested priority and deps}

Suggested Execution Batches:
  Batch 1 (parallel): [{story IDs}] — {what they do}
  Batch 2 (parallel): [{story IDs}] — {what they do}
  ...

Estimated Iterations: {count} (assuming 1 story per iteration)
High-Risk Stories (may need human intervention): [{story IDs}]

Phase 4: PRD Amendments (User Approval Required)

Present the consolidated report and ask:

1. "Apply MUST-FIX amendments to the PRD?" (recommended)
2. "Apply MUST-FIX + SHOULD-FIX?"
3. "Add missing stories from gap analysis?"
4. "Just report, I'll fix manually"

If approved, directly edit the prd.json:

• Fix dependency chains
• Split oversized stories
• Add missing stories
• Improve acceptance criteria
• Correct file references
• Update priorities for optimal sequencing

After amendments:

• Re-run Phase 1 structural validation to confirm no new issues
• Report the diff (what changed in the PRD)

Key Principles

1. Catch plan bugs before they become code bugs — a wrong dependency in the PRD wastes an entire Ralph iteration
2. Agents read actual code, not just the plan — the Codebase Reality Agent verifies every assumption against the real files
3. File collisions are the #1 Ralph killer — two stories editing the same function without a dependency chain will produce merge conflicts
4. One-context-window stories — if a story can't fit in one iteration, Ralph will produce incomplete work
5. External dependencies block Ralph — flag anything that needs human action (copying data from another repo, getting API keys, manual deploys)
6. Amend the PRD, don't just report — the goal is a PRD that Ralph can execute cleanly