Bundle Curator

bundle-curator

A workflow skill for filling pipeline-generated bundle stubs with curated scholarly content.

Each invocation targets one book directory (skills/<skill>/references/<book>/), reads the book's chunk files and metadata, drafts a README.md and key-passages.md, presents the draft at a blocking human-review gate, and writes approved content back using the pipeline's guard_or_write primitive. An opt-in wrapper can then run the downstream verification gates (pipeline stitch --check, pipeline pkg).

Scope boundary (D-01, D-02): this skill fills stubs in already-bundled books. It never runs pipeline ingest or pipeline bundle — upstream conversion and bundling are always the user's job. On approval it DOES finalize the curated book itself: it marks the files in that book's metadata.yaml human_edited list and commits them with a tightly-scoped git add + git commit (curated files + that one metadata.yaml only — never git add -A). This finalize-on-approval behavior supersedes the original D-04/D-13 "remind only" rule by user directive (2026-06-27).

When to invoke this skill

Invoke when the user asks to:

• Curate a book, fill stubs, fill _TODO: lines, or fill [title TBD] entries
• Draft README.md or key-passages.md for an ingested book
• Run downstream verify gates (stitch --check or pkg) on curated content
• Review curation output and write it back to the bundle directory

Do not invoke to re-ingest books or force-rebundle — those remain the user's job.

Before you start (prerequisites)

1. The book must be bundled. Both skills/<skill>/references/<book>/README.md and key-passages.md must exist (created by pipeline bundle <skill> --book <book>). If either file is absent, STOP:

   "The book <book> has not been bundled yet. Please run: pipeline bundle <skill> --book <book> then invoke this skill again." Do not run pipeline bundle yourself.

2. Identify the target. Know which skill and book directory you are curating. Confirm with the user if ambiguous.

Workflow

Step 1: Discover stubs → Step 2: Detect format → Step 3: Read source
→ Step 4: Draft → Step 5: Anti-confabulation check
→ Step 6: BLOCKING review gate → Step 7: Write-back + mark human_edited + commit
→ Step 8: Opt-in verify

Step 1: Discover stubs

Search the book's stub files for unfilled pipeline markers:

grep -c "_TODO:\|\[title TBD\]" "skills/<skill>/references/<book>/README.md"
grep -c "_TODO:\|\[title TBD\]" "skills/<skill>/references/<book>/key-passages.md"

• Both files absent → prerequisite-not-met (see "Before you start").
• Grep count is 0 in both → book already curated; confirm with user before redoing.
• Any count > 0 → proceed; note the stub count for the review gate.

Step 2: Detect format

Infer primary parallel-translation vs edited anthology vs secondary monograph from four signal tiers:

Tier 1 — metadata.yaml translations list (most reliable): If translations is a list with ≥ 2 entries → primary.

Tier 2 — Chunk frontmatter section locators: Read each chunks/*.md frontmatter. If any chunk has section not null or source_label set → primary.

Tier 3 — Anthology signal: If neither Tier 1 nor Tier 2 fires, check metadata.author for an (ed.) / (eds.) suffix — the primary signal for an edited volume. Corroborate with a scan of chunks/*.md for multiple distinct chapter headings or bylines naming different people. The pipeline emits no dedicated anthology field, so when the (ed.)/(eds.) suffix is absent but the book still reads like a collection (multiple bylines, an editor's preface, disparate essay titles), do not guess — propose ANTHOLOGY at the Step 6 gate with the basis stated plainly and let the user confirm or override. → anthology.

Tier 4 — Default: None of Tiers 1–3 present → secondary monograph.

Record the inferred format and the tier that triggered it. This is proposed to the user at the Step 6 gate and can be overridden — it is never applied silently (D-05).

Step 3: Read source material

Read all of the following before drafting:

• skills/<skill>/references/<book>/chunks/*.md — verbatim passage source. Every blockquote in your draft must come from these files, character-for-character.
• skills/<skill>/references/<book>/metadata.yaml — title, author, year, source, human_edited list, and translations. Handle null fields per the Step 5 null-metadata rule.
• skills/<skill>/references/<book>/.bundle-prompt.md — the pipeline's per-book curation contract (D-12). Read it to understand what the pipeline expects for this book.

Step 4: Draft content

Before drafting, read the relevant format guide and locator reference:

• Primary format → read references/format-primary.md
• Anthology format → read references/format-anthology.md
• Secondary format → read references/format-secondary.md
• All books → read references/locator-forms.md for locator syntax rules

Draft README.md: Fill the _TODO: stub sections using the format guide's README skeleton. Sections to fill: ## Place in the System, ## Key Terms, ## How to Read This, and (primary only) ## Translation Notes.

Draft key-passages.md: One H2 entry per chunk stub. For each entry:

1. Derive the locator via the priority chain: source_label → §N/¶N from section → chunk:NN-slug from the chunk filename (strip .md). See references/locator-forms.md.
2. Choose a representative verbatim passage (≥ 20 chars) from the chunk.
3. Copy the passage character-for-character as a > blockquote — never paraphrase.
4. Write the Note per the format guide: 1–2 sentences for primary; three-point (claim / context / cross-reference) for secondary.
5. Replace [title TBD] with a concise descriptive title.

Never invent §N locators where chunk frontmatter has section: null.

Step 5: Anti-confabulation check

Before presenting the draft, run this mechanical check on every > blockquote in the drafted key-passages.md:

1. Strip the > prefix from each blockquote line; concatenate the full passage text.
2. Search all chunks/*.md for a continuous substring of ≥ 20 characters from the passage text.
3. If no chunk contains any ≥ 20-char substring → FLAG the blockquote as unverifiable. Insert this comment above the flagged entry in the draft: <!-- UNVERIFIABLE: no 20-char substring match found in chunks — verify before approving -->
4. List all flagged passages in the Step 6 gate presentation.

Null-metadata rule: If title, author, or year is null in metadata.yaml, write:

*[not recoverable from ingested metadata — verify against physical copy]
— by [Author not recoverable from ingested metadata], [year unknown].*

Do not supply values from training data, even if the work seems identifiable from its chunks or context.

Step 6: Present draft (BLOCKING review gate)

Present ALL of the following to the user before writing anything:

1. Full draft — the complete README.md and key-passages.md as you intend to write them.
2. Proposed format — state PRIMARY, ANTHOLOGY, or SECONDARY and the tier that triggered it:

   "Detected format: SECONDARY MONOGRAPH. Basis: no §/¶ locators in chunks; no translations list in metadata. Override? [primary / anthology / secondary / confirm]"

   "Detected format: ANTHOLOGY. Basis: metadata.author reads 'Marina F. Bykova (ed.)'. Override? [primary / anthology / secondary / confirm]"

3. Flagged passages — any blockquotes that failed the ≥ 20-char substring check, clearly marked as potentially unverifiable.

WAIT for the user's approve, revise, or veto response before writing anything. Do not advance to write-back or to the next book without explicit input. If the user revises, incorporate changes and re-present if the edits are significant. State plainly that approve will finalize the book — write the files, mark them in human_edited, and commit them (Step 7) — so the approval is informed.

This gate is non-negotiable (D-03) — it mirrors the Phase 9/10 human-review discipline.

Step 7: Write-back, mark human_edited, and commit (on approval)

Approval at the Step 6 gate is the trigger to finalize the book. Do all three sub-steps in order — do not stop after the write.

7a — Write the files. Write curated content back using guard_or_write. Read the human_edited list from metadata.yaml, then call:

guard_or_write(
    content=<drafted text>,
    target=Path("skills/<skill>/references/<book>/README.md"),
    book_dir=Path("skills/<skill>/references/<book>"),
    human_edited=<list from metadata.yaml>,
    force=False
)

Repeat for key-passages.md. Never write files outside book_dir.

If guard_or_write raises HumanEditedRefusalError (a file is already in human_edited), the book was already curated: report the .new sidecar path and STOP — do not force-overwrite and do not proceed to 7b/7c.

7b — Mark human_edited (approval is the proof; supersedes D-13 per user directive 2026-06-27). Add "README.md" and "key-passages.md" to the human_edited list in that book's metadata.yaml, preserving the rest of the file (round-trip with sort_keys=False):

import yaml
from pathlib import Path
m = Path("skills/<skill>/references/<book>/metadata.yaml")
d = yaml.safe_load(m.read_text())
he = d.get("human_edited") or []
for f in ("README.md", "key-passages.md"):
    if f not in he:
        he.append(f)
d["human_edited"] = he
m.write_text(yaml.safe_dump(d, sort_keys=False, allow_unicode=True, default_flow_style=False))

This protects the curated files from any future guard_or_write / pipeline bundle clobber. NEVER run pipeline ingest at any point — its completion flush would overwrite this edit (manifest-clobber hazard; see project MEMORY.md).

7c — Commit (scoped; supersedes D-04 per user directive 2026-06-27). Stage and commit ONLY the two curated files and this book's metadata.yaml. NEVER git add -A / git add . — the repo may hold unrelated untracked files.

git add "skills/<skill>/references/<book>/README.md" \
        "skills/<skill>/references/<book>/key-passages.md" \
        "skills/<skill>/references/<book>/metadata.yaml"
git commit -m "content(<skill>): curate <book> + mark human_edited"

Run no other git command (no add -A, push, branch, rebase, or reset). The .skill build artifact under dist/ is gitignored and reproducible via pipeline pkg — do not force-add it. Report the commit hash to the user.

Step 8: Opt-in verify (user-initiated)

Run downstream verification gates only on explicit user request — never automatically.

# Size-gate dry-run (no file modified; exit 0 = OK, exit 10 = too large)
pipeline stitch <skill> --check

# Locator-lint + size gate + ZIP (exit 0 = OK, exit 37 = BUND-05 lint fail)
pipeline pkg <skill>

Report exit codes and full stdout/stderr to the user. Common exit codes:

• 0 — success
• 10 — SKILL.md exceeds 5120-byte cap
• 37 — BUND-05 locator-lint failure (invalid H2 in a key-passages.md)
• 32 / 33 — stitch error (SKILL.md missing or pipeline marker pair absent)

Never run pipeline ingest or pipeline bundle --force — upstream operations remain the user's job (D-02) regardless of request. (The only git this skill runs is the scoped git add + git commit in Step 7c; never run a broader git command such as git add -A, git push, or any branch/rebase/reset.)

Anti-patterns to avoid

• Re-running pipeline bundle to discover stubs. Use grep (Step 1) instead. Bundle may silently overwrite partially-curated stubs not yet in human_edited.
• Marking human_edited before approval. Only add the curated files to human_edited (and commit) AFTER the human approves the draft at the Step 6 gate — approval is the trigger. Never mark stubs or an unapproved draft as human-edited.
• Fabricating bibliographic info. When metadata fields are null, use the null-metadata template from Step 5. Never fill from training data.
• Single-digit chunk locators. chunk:3-slug FAILS BUND-05; chunk:03-slug is valid. Use the chunk filename's index as-is (the pipeline zero-pads to 3 digits).
• Attributing an anthology passage to the volume editor, or to the contributor on the wrong side of an intra-chunk chapter boundary. Anthology chunks routinely hold the tail of one essay and the head of the next; verify the split point (references/ format-anthology.md §6) before naming a contributor in the Note.
• Inventing §N locators for section-less books. Chunks with section: null and no source_label must use chunk:NN-slug — never fabricate section numbers.
• Curating a degenerate mega-chunk book without warning. If a book has 1 chunk > ~1000 lines, note it at the Step 6 gate: the user may want to re-ingest with pipeline ingest --force (their job per D-02) before curation to get meaningful per-chunk entries.
• Broad git operations. Step 7c commits ONLY the two curated files + that book's metadata.yaml. Never git add -A / git add . (the repo may hold unrelated untracked files), and never git push, branch, rebase, or reset.
• Running pipeline ingest or pipeline bundle --force. Upstream conversion and bundling remain the user's job (D-02).

Reference files

These files are loaded on demand at the steps indicated. The SKILL.md body tells you when to read each one — do not pre-load all four for every invocation.