Repo Codebook Generator

This skill is a boundary/artifact generator. It produces a single, versioned repository snapshot document.

Output artifacts

• Codebook: docs/artifacts/repo_codebook.md
• Persistent config (state): docs/artifacts/repo_codebook.config.json
• Rationale: these are generated documentation artifacts, not production source.

Non-negotiables

• Do NOT include secrets or env files (e.g., .env, .env.*).
• Exclude generated/build/runtime artifacts and other non-source noise.
• File descriptions must be 1 line max, objective, and accurate.
• Skip empty/whitespace-only files in the code section.
• If you add comments to code while generating/fixing: ONLY essential comments, in ENGLISH.

Persistent config (stateful)

The generator maintains a stateful config file so users can add more ignores without editing .gitignore or the skill code.

• Path: docs/artifacts/repo_codebook.config.json
• Behavior: created automatically on first run if missing (bootstrapped from the skill template when available).

Config fields

• version: config schema version (integer).
• codebook_version: last generated codebook version (semver string, e.g., 1.0.7). Used to persist versioning even if repo_codebook.md is deleted.
• ignore_globs_extra: additional glob patterns to exclude (e.g., data/**, *.pdf).
• skip_empty_files: if true, empty/whitespace-only files are omitted from the code section.
• max_text_file_bytes: maximum size for text files to include (bytes).

How to run (recommended)

Run from the repository root you want to document:

uv run python ~/.codex/skills/repo-codebook-generator/scripts/generate_repo_codebook.py --repo-root "$PWD"

Non-interactive mode (CI / automation)

To skip prompts and generate immediately:

uv run python ~/.codex/skills/repo-codebook-generator/scripts/generate_repo_codebook.py --repo-root "$PWD" --non-interactive

Manage ignores (recommended)

Add patterns:

uv run python ~/.codex/skills/repo-codebook-generator/scripts/generate_repo_codebook.py --repo-root "$PWD" --add-ignore "data/**" --add-ignore "*.pdf"

Remove patterns:

uv run python ~/.codex/skills/repo-codebook-generator/scripts/generate_repo_codebook.py --repo-root "$PWD" --remove-ignore "*.pdf"

Update config only (no generation):

uv run python ~/.codex/skills/repo-codebook-generator/scripts/generate_repo_codebook.py --repo-root "$PWD"  --config-only --add-ignore "out/**"

What counts as "generated/build/runtime artifacts"

Common examples: .env, .venv, __pycache__, .mypy_cache, .ruff_cache, .pytest_cache, *.egg-info, .coverage, htmlcov, lockfiles, etc.

Steps (must follow in order)

1) Ensure output directory exists

Create:

• docs/artifacts/

2) Load persistent config

Read (or create) docs/artifacts/repo_codebook.config.json and apply:

• built-in excludes + ignore_globs_extra
• file-size threshold (max_text_file_bytes)
• empty-file behavior (skip_empty_files)
• persisted codebook_version (for version continuity)

3) Interactive preflight (before generation)

By default (when running in a TTY), the generator runs an interactive preflight before writing docs/artifacts/repo_codebook.md:

1. Print an "Ignore Summary" showing:

   • Layer 1: .gitignore / git excludes behavior
   • Layer 2: built-in excludes (components + globs)
   • Layer 3: current ignore_globs_extra from config

2. Prompt with enumerated choices:

   1. Add more files/folders/patterns to ignore (persistent)
   2. Continue without changes

3. If the user chooses 1, accept multiple entries (one per line; empty line ends).

   • Directories are canonicalized and persisted as dir/** (ignore the directory and all descendants)
   • Globs like *.pdf are kept as-is

4. After saving, print what was added and show the full ignore_globs_extra list.

5. Prompt again:

   1. Generate repo_codebook.md now
   2. Add more ignores (loops back)

Note:

• Use --non-interactive to disable prompts (CI / automation).

4) Generate project structure (tree) respecting .gitignore

Preferred: use tree --gitignore plus extra excludes via -I.

Recommended command (no colors, include dot dirs, directories first):

bash skills/repo-codebook-generator/scripts/get_tree.sh

Notes:

• --gitignore ensures .gitignore rules are applied.
• Extra ignores from config are applied best-effort (converted to a tree -I expression via IGNORE_PATTERN_EXTRA).
• If tree is not installed, the generator falls back to a find-based listing (best-effort).

5) Build the file list to document (matching tree semantics)

Use Git as the source of truth for "not ignored":

• git ls-files -co --exclude-standard

Then apply:

• built-in excludes
• persistent ignore_globs_extra from config

Directory semantics:

• Directory-like ignore entries are expanded to exclude both the directory itself and all descendants (e.g., data -> data and data/**) so directory pruning works correctly.

6) Write / update docs/artifacts/repo_codebook.md

The document must contain:

## Project Info
- name: <short representative name>
- description:
  - <bullet 1>
  - <bullet 2>
- codebook_version: <semver>

## Project Structure
```bash
<tree output>
```

### Descriptions
- <path>: <one-line objective description>
...

## Project Current Code
```<path>
<full file contents>
```

...

7) Versioning rule for codebook_version

• If the file is created for the first time: 1.0.0
• If it already exists: bump PATCH by default (e.g., 1.0.0 -> 1.0.1)
• If repo_codebook.md is missing but config contains codebook_version, bump PATCH from config to preserve continuity
• Only bump MINOR/MAJOR if explicitly requested.
• After successful generation, persist the new codebook_version into docs/artifacts/repo_codebook.config.json.

8) Size / binary / empty-file safety

• Skip binary files and very large files (default threshold: 512 KB or max_text_file_bytes from config) and add a note like:
  • - <path>: skipped (binary or too large)
• Empty/whitespace-only files:
  • Descriptions show skipped (empty file)
  • Code blocks are omitted (when skip_empty_files=true)

How to run (recommended)

Generate the codebook:

uv run python ~/.codex/skills/repo-codebook-generator/scripts/generate_repo_codebook.py --repo-root "$PWD"

This will:

• Ensure docs/artifacts/ exists
• Ensure docs/artifacts/repo_codebook.config.json exists (create if missing, bootstrapped from template when possible)
• Run an interactive ignore preflight (unless --non-interactive is used)
• Produce tree output using .gitignore + built-in excludes + config excludes (best-effort)
• Generate/update the codebook with bumped patch version (persisted in config for continuity)
• Include one-line per file + full code blocks (excluding empty/binary/too-large)

Converted and distributed by TomeVault — claim your Tome and manage your conversions.