Generate Topic
Orchestrates the full lesson generation pipeline for one topic. Ensures every downstream step (jargon, quiz, reference, SVG theming, verification) actually happens — not just the lesson writing.
When to use
- Generating a new topic from a MAP.md
- "Completing" an existing topic that's missing artifacts (run against it to fill gaps)
- Any time you want the full pipeline, not just a quick lesson draft
Input
- A workspace path (e.g.,
library/oidc-rust)
- A topic slug from the workspace's MAP.md (e.g.,
token-validation-middleware)
- The topic's prereqs should already be complete
The Pipeline (4 Phases)
Phase 1: Research (PARALLEL — fan out 2-4 subagents)
Dispatch simultaneously:
| Agent |
Task |
Output |
| Domain research |
Web search for topic concepts, key facts, 3+ sources |
.scratch/research/{slug}.md |
| Workspace context |
Read MAP.md, existing lessons, RESOURCES.md — what's taught, what prereqs cover, avoid repetition |
.scratch/research/{slug}-context.md |
| Source verification |
Check top URLs from RESOURCES.md are live, extract key claims relevant to this topic |
.scratch/research/{slug}-sources.md |
Source-ingested topics: If source-chunks/{domain}.json exists, this topic was derived from a source document. In this case:
- Skip web research — the source chunks ARE the research
- Read the chunk(s) matching this topic's heading from the JSON
- Use chunk content as the authoritative source for the lesson
- When writing SR questions, populate
source_section (chunk heading), source_page (chunk page_start), and source_quote (exact passage from the chunk content) on every card
After all return: Synthesize in main context. Resolve conflicts between sources. Determine: what to teach, what to cite, what the learner already knows from prereqs.
Concept hints (opt-in): If source-chunks/{domain}.json exists, run:
python tools/concept_hints.py source-chunks/{domain}.json --topic {slug} --domain {domain}
This produces .scratch/concepts/{slug}.json with:
- Ranked candidate glossary terms (use as starting checklist for jargon, not a mandate)
- L-level suggestions per concept (L1=core recall, L2=practice, L3=analysis — informed by foundational-ness score + prerequisite depth)
- Prerequisite edge suggestions (use for "explain why X depends on Y" question framing)
If no source-chunks exist (web-researched topic), skip this step — the agent uses its own judgment for terms and levels.
Failure handling: If 1 of 3 returns empty, retry once. If still empty after retry, proceed with available research (note the gap).
Phase 2: Generate (SEQUENTIAL — main context)
Before writing: Check the topic's current status in the per-user overlay (via the
running server's GET /api/map/{domain}/{slug}/status, or tools/lib/overlay.py
Overlay(workspace).get(node_id)). If it's complete, this is a rewrite:
- Reset status to
in-progress in the overlay (POST .../status with in-progress,
or Overlay(workspace).set(node_id, "in-progress")) — status is per-user overlay
state, NEVER written into the committed MAP.md
- Delete the existing quiz page for this topic (will be regenerated below)
- Remove existing SR questions for this topic from the domain JSONL (filter out lines where
"topic": "{slug}")
- Regenerate the map page (reflects the overlay status change immediately)
- Inform the user: "Topic was complete — reset to in-progress for rewrite. Mark complete again once you've reviewed the new lesson."
Note: The pipeline does NOT auto-mark the topic complete after generation. The user marks it complete themselves (via the lesson page button) after reviewing the rewritten content.
Each step depends on the previous:
- Body content only — produce ONLY the lesson body (h2 sections, paragraphs, SVGs, tables, exercises). Do NOT write
<!DOCTYPE>, <html>, <head>, script tags, or import statements. The page_template.py handles all boilerplate.
- Write the lesson — using synthesized research. Follow teach skill conventions (SVG diagram with CSS vars, citations, key-concept blocks, exercise with hint + answer). The exercise tests the lesson's Win statement — core concept comprehension, not detail recall or gotchas. For exercise design and the decision/callout hierarchy (including the required "When to use which" criteria when presenting alternatives), follow
visual-teaching.md (§ Exercise Design, § Callout Hierarchy) — the owner of those rules. Call python3 -c "from tools.lib.page_template import render_lesson_page; ..." or have the agent write the body to a temp file and wrap with the template.
- Write the reference doc — produce body content only (tables, lists, one-sentence summaries). Wrap with
render_reference_page() from tools/lib/page_template.py.
- Write SR questions — append to
learning-records/questions/{domain}.jsonl. 4-8 open-answer questions + 2-3 interactive questions per topic. Follow the archetypes, criteria format, eli5, provenance/level tagging, and difficulty ordering in references/sr-question-authoring.md — the single owner of SR-question rules.
- Extract code files — For each unique
data-file in the lesson HTML, write the final-state version to reference/code/{lesson-slug}/. If a file appears in multiple blocks (complete → diff), assemble the final version by applying diffs in document order. Include a README.md listing each file. Add a "Code Files" section to the lesson body (before "What's Next") with <a href="..." download> links. Skip this step if the lesson has no data-file blocks.
- Update previous lesson's forward link — If this topic has a prereq that's already complete, find that lesson's "What's Next" section and replace the plain-text topic reference with an
<a href="{new-lesson-filename}"> link. This connects the reading flow so learners can navigate forward without returning to the map.
- Generate quiz page —
python3 tools/generate-quiz-page.py --workspace {workspace} --lesson-id {slug} --title "{title}" --lesson-file {filename} --map-page {map-page} --domain "{domain}" --domain-slug {domain-slug}
Phase 3: Post-process (PARALLEL — fan out 2-3 subagents)
Dispatch simultaneously:
| Agent |
Task |
Blocking? |
| Jargon annotation |
Read lesson, find glossary-data keys, wrap first use of each term with <span class="term" data-term="KEY"> |
Optional (skip if fails) |
| SVG variable check |
python3 tools/check-svg-vars.py --workspace {workspace} — report hardcoded hex |
Required (fix before verify) |
| SR quality check |
mise run sr:check -- {slug} — report prompt issues, missing criteria |
Optional (report only) |
After return: If SVG check found violations, fix them (sed replacement). If jargon returned annotations, apply them.
Phase 4: Verify (PARALLEL — fan out 2-3 subagents)
Dispatch simultaneously:
| Agent |
Role |
Gate |
| Link + lint check |
mise run verify |
MUST pass |
| Visual check |
Browser: navigate to lesson URL, confirm SVG visible in dark mode, theme toggle works, bottom nav present, glossary tooltips appear |
MUST pass |
| Structural compliance |
python3 tools/check-topic-completeness.py --workspace {workspace} --topic {slug} |
MUST pass |
| Reference project |
If a reference project exists for this track: copy code to project, validate via toolchain (compile, import, lint). See ADR 0010. |
MUST pass (if applicable) |
Gate: ALL must pass. Any failure blocks the topic from being marked complete.
After all pass:
- Update MAP.md — add the
lesson_file: field for the topic (committed graph data).
Do NOT write status into MAP.md — status is per-user overlay state. Set completion
in the overlay: if this is a new topic, mark it complete in the overlay
(Overlay(workspace).set(node_id, "complete"), or the user marks it via the lesson
page button). If this is a rewrite (reset to in-progress in Phase 2), leave the
overlay at in-progress — the user marks complete after reviewing.
- Regenerate map page —
python3 tools/generate_map_page.py {map.MAP.md} --workspace {workspace} --output {workspace}/lessons/{domain}-map.html
- Regenerate index —
python3 tools/generate_index_page.py --scan-dir examples
Running Against Existing Topics
The pipeline handles existing topics differently based on whether content changes:
Verification-only (no --force, files exist):
- Research phase: skips (or refreshes if sources are stale)
- Generate phase: skips (files exist, no content change)
- Post-process phase: re-checks (jargon overwrites existing spans safely)
- Verify phase: always runs (confirms current state is compliant)
- Status: NOT reset — no content changed, no reason to un-complete
Rewrite (--force or user explicitly asks to regenerate):
- Status resets to
in-progress at the start of Phase 2
- Old quiz page and SR questions for this topic are deleted
- Full generation runs (new lesson, ref, quiz, SR)
- Status remains
in-progress after pipeline completes
- User marks complete after reviewing the new content
This is how you audit existing content: generate-topic --workspace X --topic Y should pass silently if everything is correct. To rewrite: ask to regenerate or use --force.
Multiple Topics
Topics generate sequentially, one at a time. Each topic completes the full 4-phase pipeline before the next begins. This ensures:
- No dilution: Each topic gets full research depth and verification attention
- Prereq awareness: Later topics can reference what earlier ones established
- Early failure detection: A broken topic blocks further generation (don't accumulate debt)
- Quality over throughput: 2 excellent topics > 5 mediocre ones
The parallel fan-out happens WITHIN each topic (research agents, verify agents) — never ACROSS topics. If you need 3 topics generated, that's 3 sequential runs of the full pipeline, not one run with 3 topics batched.
Topic 1: research (parallel) → generate → post-process (parallel) → verify (parallel) → ✓ complete
Topic 2: research (parallel) → generate → post-process (parallel) → verify (parallel) → ✓ complete
Topic 3: research (parallel) → generate → post-process (parallel) → verify (parallel) → ✓ complete
Does NOT
- Replace the teach skill (teach is the creative engine; this is the assembly line)
- Add infrastructure (no workflow YAML parser, just a skill document)
- Force the user to use it (teach works standalone for quick lessons)
- Generate content for topics whose prereqs aren't complete
- Mark topics complete without the verify gate passing
Error Recovery
| Failure |
Response |
| Research agent returns empty |
Retry once. If still empty, proceed with partial research + note gap. |
| Lesson generation produces bad HTML |
mise run verify catches broken links/structure in Phase 4. |
| Jargon pass corrupts file |
Non-blocking. If term count = 0 after annotation, revert to pre-jargon state. |
| Quiz generation fails (no questions) |
Blocking. Can't have a quiz without questions. Write questions first, then retry. |
| Verify agent disagrees with lint |
Trust the tool output (lint/completeness scripts), not the visual check. Fix what the tool says. |
| Playwright can't connect |
Skip visual check, report gap. Other two verify agents still gate. |
1---2name: generate-topic3description: Generate a complete topic: research → lesson → post-process → verify. Fans out subagents for research and verification. Trigger: generate topic, generate lesson, complete topic, full generation.4---56# Generate Topic78Orchestrates the full lesson generation pipeline for one topic. Ensures every downstream step (jargon, quiz, reference, SVG theming, verification) actually happens — not just the lesson writing.910## When to use1112- Generating a new topic from a MAP.md13- "Completing" an existing topic that's missing artifacts (run against it to fill gaps)14- Any time you want the full pipeline, not just a quick lesson draft1516## Input1718- A workspace path (e.g., `library/oidc-rust`)19- A topic slug from the workspace's MAP.md (e.g., `token-validation-middleware`)20- The topic's prereqs should already be complete2122## The Pipeline (4 Phases)2324### Phase 1: Research (PARALLEL — fan out 2-4 subagents)2526Dispatch simultaneously:2728| Agent | Task | Output |29|-------|------|--------|30| Domain research | Web search for topic concepts, key facts, 3+ sources | `.scratch/research/{slug}.md` |31| Workspace context | Read MAP.md, existing lessons, RESOURCES.md — what's taught, what prereqs cover, avoid repetition | `.scratch/research/{slug}-context.md` |32| Source verification | Check top URLs from RESOURCES.md are live, extract key claims relevant to this topic | `.scratch/research/{slug}-sources.md` |3334**Source-ingested topics:** If `source-chunks/{domain}.json` exists, this topic was derived from a source document. In this case:35- **Skip web research** — the source chunks ARE the research36- Read the chunk(s) matching this topic's heading from the JSON37- Use chunk content as the authoritative source for the lesson38- When writing SR questions, populate `source_section` (chunk heading), `source_page` (chunk page_start), and `source_quote` (exact passage from the chunk content) on every card3940**After all return:** Synthesize in main context. Resolve conflicts between sources. Determine: what to teach, what to cite, what the learner already knows from prereqs.4142**Concept hints (opt-in):** If `source-chunks/{domain}.json` exists, run:43```44python tools/concept_hints.py source-chunks/{domain}.json --topic {slug} --domain {domain}45```46This produces `.scratch/concepts/{slug}.json` with:47- Ranked candidate glossary terms (use as starting checklist for jargon, not a mandate)48- L-level suggestions per concept (L1=core recall, L2=practice, L3=analysis — informed by foundational-ness score + prerequisite depth)49- Prerequisite edge suggestions (use for "explain why X depends on Y" question framing)5051If no source-chunks exist (web-researched topic), skip this step — the agent uses its own judgment for terms and levels.5253**Failure handling:** If 1 of 3 returns empty, retry once. If still empty after retry, proceed with available research (note the gap).5455### Phase 2: Generate (SEQUENTIAL — main context)5657**Before writing:** Check the topic's current status in the per-user overlay (via the58running server's `GET /api/map/{domain}/{slug}/status`, or `tools/lib/overlay.py`59`Overlay(workspace).get(node_id)`). If it's `complete`, this is a **rewrite**:601. Reset status to `in-progress` in the overlay (`POST .../status` with `in-progress`,61 or `Overlay(workspace).set(node_id, "in-progress")`) — status is per-user overlay62 state, NEVER written into the committed MAP.md632. Delete the existing quiz page for this topic (will be regenerated below)643. Remove existing SR questions for this topic from the domain JSONL (filter out lines where `"topic": "{slug}"`)654. Regenerate the map page (reflects the overlay status change immediately)665. Inform the user: "Topic was complete — reset to in-progress for rewrite. Mark complete again once you've reviewed the new lesson."6768**Note:** The pipeline does NOT auto-mark the topic complete after generation. The user marks it complete themselves (via the lesson page button) after reviewing the rewritten content.6970Each step depends on the previous:71721. **Body content only** — produce ONLY the lesson body (h2 sections, paragraphs, SVGs, tables, exercises). Do NOT write `<!DOCTYPE>`, `<html>`, `<head>`, script tags, or import statements. The `page_template.py` handles all boilerplate.732. **Write the lesson** — using synthesized research. Follow teach skill conventions (SVG diagram with CSS vars, citations, key-concept blocks, exercise with hint + answer). The exercise tests the lesson's **Win statement** — core concept comprehension, not detail recall or gotchas. For exercise design and the decision/callout hierarchy (including the required "When to use which" criteria when presenting alternatives), follow `visual-teaching.md` (§ Exercise Design, § Callout Hierarchy) — the owner of those rules. Call `python3 -c "from tools.lib.page_template import render_lesson_page; ..."` or have the agent write the body to a temp file and wrap with the template.743. **Write the reference doc** — produce body content only (tables, lists, one-sentence summaries). Wrap with `render_reference_page()` from `tools/lib/page_template.py`.754. **Write SR questions** — append to `learning-records/questions/{domain}.jsonl`. 4-8 open-answer questions + 2-3 interactive questions per topic. Follow the archetypes, criteria format, `eli5`, provenance/level tagging, and difficulty ordering in [references/sr-question-authoring.md](./references/sr-question-authoring.md) — the single owner of SR-question rules.765. **Extract code files** — For each unique `data-file` in the lesson HTML, write the final-state version to `reference/code/{lesson-slug}/`. If a file appears in multiple blocks (complete → diff), assemble the final version by applying diffs in document order. Include a README.md listing each file. Add a "Code Files" section to the lesson body (before "What's Next") with `<a href="..." download>` links. Skip this step if the lesson has no `data-file` blocks.776. **Update previous lesson's forward link** — If this topic has a prereq that's already complete, find that lesson's "What's Next" section and replace the plain-text topic reference with an `<a href="{new-lesson-filename}">` link. This connects the reading flow so learners can navigate forward without returning to the map.787. **Generate quiz page** — `python3 tools/generate-quiz-page.py --workspace {workspace} --lesson-id {slug} --title "{title}" --lesson-file {filename} --map-page {map-page} --domain "{domain}" --domain-slug {domain-slug}`7980### Phase 3: Post-process (PARALLEL — fan out 2-3 subagents)8182Dispatch simultaneously:8384| Agent | Task | Blocking? |85|-------|------|-----------|86| Jargon annotation | Read lesson, find glossary-data keys, wrap first use of each term with `<span class="term" data-term="KEY">` | Optional (skip if fails) |87| SVG variable check | `python3 tools/check-svg-vars.py --workspace {workspace}` — report hardcoded hex | Required (fix before verify) |88| SR quality check | `mise run sr:check -- {slug}` — report prompt issues, missing criteria | Optional (report only) |8990**After return:** If SVG check found violations, fix them (sed replacement). If jargon returned annotations, apply them.9192### Phase 4: Verify (PARALLEL — fan out 2-3 subagents)9394Dispatch simultaneously:9596| Agent | Role | Gate |97|-------|------|------|98| Link + lint check | `mise run verify` | MUST pass |99| Visual check | Browser: navigate to lesson URL, confirm SVG visible in dark mode, theme toggle works, bottom nav present, glossary tooltips appear | MUST pass |100| Structural compliance | `python3 tools/check-topic-completeness.py --workspace {workspace} --topic {slug}` | MUST pass |101| Reference project | If a reference project exists for this track: copy code to project, validate via toolchain (compile, import, lint). See ADR 0010. | MUST pass (if applicable) |102103**Gate:** ALL must pass. Any failure blocks the topic from being marked complete.104105**After all pass:**1061. Update MAP.md — add the `lesson_file:` field for the topic (committed graph data).107 Do NOT write `status` into MAP.md — status is per-user overlay state. Set completion108 in the overlay: if this is a **new topic**, mark it `complete` in the overlay109 (`Overlay(workspace).set(node_id, "complete")`, or the user marks it via the lesson110 page button). If this is a **rewrite** (reset to `in-progress` in Phase 2), leave the111 overlay at `in-progress` — the user marks complete after reviewing.1122. Regenerate map page — `python3 tools/generate_map_page.py {map.MAP.md} --workspace {workspace} --output {workspace}/lessons/{domain}-map.html`1133. Regenerate index — `python3 tools/generate_index_page.py --scan-dir examples`114115## Running Against Existing Topics116117The pipeline handles existing topics differently based on whether content changes:118119**Verification-only (no `--force`, files exist):**120- Research phase: skips (or refreshes if sources are stale)121- Generate phase: skips (files exist, no content change)122- Post-process phase: re-checks (jargon overwrites existing spans safely)123- Verify phase: always runs (confirms current state is compliant)124- **Status: NOT reset** — no content changed, no reason to un-complete125126**Rewrite (`--force` or user explicitly asks to regenerate):**127- Status resets to `in-progress` at the start of Phase 2128- Old quiz page and SR questions for this topic are deleted129- Full generation runs (new lesson, ref, quiz, SR)130- Status remains `in-progress` after pipeline completes131- **User marks complete** after reviewing the new content132133This is how you audit existing content: `generate-topic --workspace X --topic Y` should pass silently if everything is correct. To rewrite: ask to regenerate or use `--force`.134135## Multiple Topics136137**Topics generate sequentially, one at a time.** Each topic completes the full 4-phase pipeline before the next begins. This ensures:1381391. **No dilution:** Each topic gets full research depth and verification attention1402. **Prereq awareness:** Later topics can reference what earlier ones established1413. **Early failure detection:** A broken topic blocks further generation (don't accumulate debt)1424. **Quality over throughput:** 2 excellent topics > 5 mediocre ones143144The parallel fan-out happens WITHIN each topic (research agents, verify agents) — never ACROSS topics. If you need 3 topics generated, that's 3 sequential runs of the full pipeline, not one run with 3 topics batched.145146```147Topic 1: research (parallel) → generate → post-process (parallel) → verify (parallel) → ✓ complete148Topic 2: research (parallel) → generate → post-process (parallel) → verify (parallel) → ✓ complete149Topic 3: research (parallel) → generate → post-process (parallel) → verify (parallel) → ✓ complete150```151152## Does NOT153154- Replace the teach skill (teach is the creative engine; this is the assembly line)155- Add infrastructure (no workflow YAML parser, just a skill document)156- Force the user to use it (teach works standalone for quick lessons)157- Generate content for topics whose prereqs aren't complete158- Mark topics complete without the verify gate passing159160## Error Recovery161162| Failure | Response |163|---------|----------|164| Research agent returns empty | Retry once. If still empty, proceed with partial research + note gap. |165| Lesson generation produces bad HTML | `mise run verify` catches broken links/structure in Phase 4. |166| Jargon pass corrupts file | Non-blocking. If term count = 0 after annotation, revert to pre-jargon state. |167| Quiz generation fails (no questions) | Blocking. Can't have a quiz without questions. Write questions first, then retry. |168| Verify agent disagrees with lint | Trust the tool output (lint/completeness scripts), not the visual check. Fix what the tool says. |169| Playwright can't connect | Skip visual check, report gap. Other two verify agents still gate. |
smileynet