Essay Pipeline

Essay Pipeline

Announce: "I'm using the essay-pipeline skill for interactive essay writing."

Architecture Overview

This pipeline uses an orchestrator-as-conductor pattern. You (the orchestrator) run all four interactive stages in your main thread, maintaining direct conversation with the user. Only non-interactive specialists are delegated as sub-agents.

User
  |
  v
essay-pipeline orchestrator (YOU -- main thread)
  |
  |-- Stage 1: Thesis Development (interactive, in your thread)
  |     Reference: references/stage-1-thesis-development.md
  |
  |-- Stage 2: Essay Structuring (interactive, in your thread)
  |     Reference: references/stage-2-essay-structuring.md
  |
  |-- Stage 3: Argument Development (interactive, per-section, in your thread)
  |     Reference: references/stage-3-argument-development.md
  |     Support: essay-fact-checker (via Task tool)
  |
  |-- Stage 4: Paragraph Writing (interactive, per-paragraph, in your thread)
  |     Reference: references/stage-4-paragraph-writing.md
  |     Support: essay-fact-checker (via Task tool)
  |     Support: essay-voice-matcher (via Task tool)
  |
  v
Final Essay Output

Why this architecture: Claude Code sub-agents cannot use AskUserQuestion. Since every stage requires interactive dialogue, the orchestrator must run all stages directly. Sub-agents are used only for non-interactive work (fact verification and voice evaluation).

State Anchoring Protocol

Start every response with a state anchor:

[Stage N/4 - {stage_name}] {status}

Examples:

• [Stage 1/4 - Thesis Development] Developing thesis through Socratic dialogue
• [Stage 2/4 - Essay Structuring] Negotiating section structure
• [Stage 3/4 - Argument Development - Section 2 of 5] Developing argument map
• [Stage 4/4 - Paragraph Writing - Section 1, Paragraph 3] Drafting paragraph

Re-anchor after:

• Navigation commands ("go back to...", "show full state")
• Stage transitions
• Resume from pause
• Any context-switching action

Tool Selection Table

Self-check rule: Before performing any action, ask: "Am I about to verify a fact or evaluate voice consistency?" If YES, delegate to the appropriate sub-agent via Task tool. NEVER verify facts or evaluate voice consistency yourself.

Delegation Mandate

Orchestrator-Owned (YOU do these)

• Session management: Create, save, restore, pause, resume sessions
• Stage transitions: Evaluate exit criteria, present summaries, get user approval
• User dialogue: ALL interactive conversation via AskUserQuestion
• Quality gate evaluation: Check exit criteria checklists
• Context assembly: Load relevant files for each stage
• Stage execution: Follow stage reference file instructions
• Final essay assembly: Combine approved paragraphs into final document
• State anchoring: Maintain stage/section/paragraph tracking in every response

Delegated to essay-fact-checker (via Task tool)

• ALL factual claim verification (Tier 1)
• ALL proactive research enrichment (Tier 2)
• Tier 3 escalation identification
• Source URL verification
• Contradictory source reporting

Delegated to essay-voice-matcher (via Task tool)

• ALL voice consistency evaluation
• Style profile analysis
• Sample essay consultation
• Profile-sample conflict detection
• Cumulative voice tracking

Never do sub-agent work yourself. If you find yourself about to look up a fact or evaluate whether text matches the user's voice, stop and delegate.

Pre-Flight Validation

Before starting the pipeline, check prerequisites:

1. Style Profile Check

Use Bash tool to check for the style profile:

test -f "{configured_path}/style-profile.md" && echo "FOUND" || echo "NOT_FOUND"

If found: Read the profile and keep it in context throughout the session.

If not found: Offer three paths via AskUserQuestion:

• (a) Quick 5-question profile: Ask the 5 express profile questions (from references/style-profile-template.md), synthesize a minimal profile, save it, and proceed.
• (b) Proceed without (degraded mode): Voice matching will be limited. The voice-matcher will work from sample essays only, or skip voice matching entirely if no samples either.
• (c) Exit to create full profile: Provide the template path and exit. User fills out the full template and returns.

2. Sample Essays Check

Use Bash/Glob to check for sample essays:

ls "{configured_path}/samples/"*.md 2>/dev/null | head -5

If found: Note the number of available samples. The voice-matcher will consult them. If not found: Warn the user. Voice matching will rely solely on the style profile.

3. WebSearch Availability

Note that WebSearch is available through the essay-fact-checker sub-agent. If fact-checker invocation fails, fall back to user-provided sources only.

Session Initialization

Check for Existing Sessions

Scan for existing sessions:

ls -d /tmp/essay-pipeline-*/ 2>/dev/null

If sessions found: Present them to the user with status summaries. Ask whether to resume an existing session or start new.

If no sessions found: Proceed to create a new session.

Create New Session

1. Generate session directory:

   mkdir -p /tmp/essay-pipeline-$(date +%Y%m%d-%H%M%S)
   

2. Initialize session state (see references/session-state-schema.md for full schema)

3. Offer pushback level selection via AskUserQuestion:

   Before we begin, how much intellectual pushback would you like?
   
   - Full (default): I'll challenge your ideas vigorously to strengthen them.
     3 rounds of pushback in thesis development, 2 in argument development.
   - Light: I'll raise concerns once and accept your response.
     1 round per point.
   - Minimal: I'll offer observations but won't challenge.
     No pushback rounds.
   
   You can change this at any time during the session.
   

4. Save initial state using atomic write protocol.

5. Begin Stage 1.

Pipeline Workflow

For each stage, follow this process:

Stage Entry

1. Read the stage reference file: references/stage-N-{name}.md
2. Anchor state: [Stage N/4 - {name}] Beginning stage
3. Follow the stage's behavioral instructions

Stage Execution

4. Conduct interactive dialogue with user per the stage's process
5. Apply stage-appropriate pushback per the configured pushback level
6. Delegate to sub-agents as needed (fact-checker in Stage 3/4, voice-matcher in Stage 4)
7. Save incremental progress to session files

Stage Exit

8. Evaluate exit criteria (from the stage reference file)
9. Present stage completion summary to the user
10. Get user approval at quality gate via AskUserQuestion
11. Save stage outputs to session directory
12. Update session state (atomic write)
13. Ask user to confirm readiness to proceed to next stage

Stage Transition

14. Drop the previous stage's reference file from active consideration
15. Keep stage OUTPUT files (thesis, outline, argument maps) as context
16. Load the next stage's reference file
17. Begin the next stage's process

Stage-Specific Execution Notes

Stage 1: Thesis Development

• Load: references/stage-1-thesis-development.md
• Follow its Socratic dialogue process
• Express path available for pre-formed theses
• Quality Gate G1: Thesis is clear, defensible, and approved

Stage 2: Essay Structuring

• Load: references/stage-2-essay-structuring.md
• Read Stage 1 output for thesis and claim type
• Negotiate structure, length, audience
• Quality Gate G2: Outline is complete and approved

Stage 3: Argument Development (Per-Section Loop)

• Load: references/stage-3-argument-development.md
• Loop through each section in the outline:

  for each section in outline:
    Develop argument map for this section
    Batch-invoke fact-checker for all claims in this section
    Handle fact-checker results with user
    Get user approval for section argument map
    Save argument map to session directory
    Update session state
  

• After all sections: Present bird's-eye review of all argument maps
• Quality Gate G3-Final: All argument maps cohere and are approved
• Optional: User can request paragraph writing for completed sections (controlled forward-jumping)

Stage 4: Paragraph Writing (Per-Paragraph Loop)

• Load: references/stage-4-paragraph-writing.md
• Loop through each section, then each paragraph:

  for each section:
    for each paragraph in section:
      Draft paragraph following argument map + style profile
      Self-audit for argument map compliance
      Present to user for approval
      Save approved paragraph
    Invoke voice-matcher on complete section
    Present section for bird's-eye review
  

• After all sections: Final voice check, final fact-check sweep, resolve deferred verifications
• Quality Gate G5: Full essay approved

Quality Gates

User Navigation

At any point, the user can issue navigation commands. Detect these in user responses and handle accordingly:

Stage Revisit Cascade

When the user goes back to an earlier stage:

1. Assess change impact (see references/session-state-schema.md):
   • Minor: Flag downstream for review
   • Moderate: Archive affected downstream artifacts
   • Major: Archive ALL downstream artifacts, warn user
2. Present impact to user: "Going back to [stage] will [impact description]. Your current work will be [preserved/archived/flagged]. Continue?"
3. If user confirms: Update session state, version old artifacts, begin the earlier stage
4. Never silently discard work

Error Handling Protocol

Graceful Degradation Hierarchy

When components fail, degrade gracefully rather than stopping:

Full capability (all components working)
  |
  v (fact-checker fails)
Degraded: User-provided sources only; claims marked DEFERRED
  |
  v (voice-matcher also fails)
Degraded: No voice checking; user self-evaluates voice
  |
  v (session state corrupt)
Recovery mode: Reconstruct from artifacts
  |
  v (artifacts also lost)
Manual recovery: Ask user where to resume

Timeout Configuration

When invoking sub-agents via Task tool, include timeout guidance in the task prompt.

Fact-Checker Invocation Protocol

During Stage 3 (Argument Development)

• When: After developing the argument map for a section
• What: Batch all factual claims in the section into one invocation
• How: Prepare claim list per the fact-checker's input format; invoke via Task tool
• Result handling: Present results to user; update argument map with verification status and sources

During Stage 4 (Paragraph Writing)

• When: When a draft paragraph introduces NEW claims not already verified in Stage 3
• What: Only new claims (do NOT re-verify claims already in argument maps)
• How: Same as Stage 3

Final Sweep

• When: After all paragraphs are approved, before Quality Gate G5
• What: All claims in the complete essay (one batch invocation)
• Purpose: Catch any claims modified during user edits, verify deferred claims

Deferred Verification Queue

• Track unverified claims in session state under deferred_verifications
• Resolve ALL deferred claims before Quality Gate G5
• Resolution: Retry verification, user provides source, revise claim, or remove claim

Task Tool Invocation Format

Task: essay-fact-checker

Verify the following claims from Section [N] of a science blog essay about [topic].

[YAML claim batch]

Return results as YAML with verification status, source URLs, and notes for each claim.
Timeout: Complete within 3 minutes.

Voice-Matcher Invocation Protocol

Invocation Schedule

• After each section is complete: Section-level voice consistency check
• After full essay is complete: Essay-level voice consistency check
• On user request: Any time the user asks about voice

Calibration (First Invocation)

On the first voice-matcher result:

1. Present the assessment to the user
2. Ask: "How accurate is this voice assessment? Rate 1-5, where 5 is very accurate."
3. Save the user's rating in session state under voice_calibration
4. Include calibration data in all subsequent voice-matcher invocations

Score Thresholds

Task Tool Invocation Format

Task: essay-voice-matcher

Evaluate the following text for voice consistency against the user's style profile.

Text to evaluate:
[text]

Style profile path: [path]
Sample essays path: [path]
Context: Stage [N], Section [N], [content type]
Previous assessments: [YAML list of prior assessments]
Calibration: [calibration data if available]
Voice reference mode: [profile | sample | hybrid]

Return assessment as YAML with score (1-5), observations, and suggestions.
Timeout: Complete within 3 minutes.

User Override Protocol

When the fact-checker's findings conflict with the user's claim:

1. Present evidence: "The fact-checker found [result]. Your claim states [claim]. Source: [URL]"
2. Offer options via AskUserQuestion:
   • Revise the claim to match verified evidence
   • Rephrase (keep the spirit, adjust specifics)
   • Keep with override (log the disagreement)
   • Provide your own source
3. Log all overrides in fact-check-log.md
4. Surface at final review: At Quality Gate G5, present all overrides for one final confirmation

Argument Map Compliance (Stage 4)

Before presenting each paragraph draft in Stage 4:

1. Self-audit: Which argument map points does this paragraph cover?
2. Tag content: Mark each sentence as MAPPED (covers an argument map point) or NOT IN MAP
3. Flag deviations: If NOT IN MAP content exists, explicitly notify the user:
   • "This paragraph includes content not in your argument map: [description]."
   • Options: (a) Keep and add to map retroactively, (b) Remove, (c) Go back to Stage 3

Context Window Management

For long essays (5,000+ words), context window pressure is a concern. Follow these rules:

Key rule: Rely on FILES for historical content, not conversation memory. After each section is complete in Stage 4, write the full section text to stage-4-draft.md and drop it from active context. When needed for continuity, read only the last paragraph of the previous section.

Session State Resilience

Atomic Writes

All session state updates use the atomic write protocol:

1. Write to .tmp file
2. Copy current state to .bak
3. Rename .tmp to primary

Incremental Saves

Save progress at granular checkpoints:

• After each completed argument point (Stage 3)
• After each approved paragraph (Stage 4)
• After each stage transition
• After any user navigation command

Recovery

On session load, attempt: Primary state -> Backup state -> Artifact reconstruction (see references/session-state-schema.md)

Pipeline Completion

When all stages are complete and Quality Gate G5 is passed:

1. Final voice check: Invoke voice-matcher on full essay
2. Final fact-check sweep: Invoke fact-checker on all claims in complete essay
3. Resolve deferred verifications: All must be resolved
4. Present completion summary:

   Pipeline Complete!
   
   Word count: [N] words
   Sections: [N]
   Sources verified: [N] claims with [M] unique sources
   Voice consistency: [score]/5
   User overrides: [N] (see fact-check-log.md)
   Deferred claims resolved: [N] of [N]
   Time spent: [duration]
   

5. Write final essay to {session_dir}/final-essay.md
6. Offer export: "Would you like me to write the essay to a specific file path?"
7. Archive session: Mark session as "completed" in state

Quick Reference: Stage Summary