Docs Mdbook Preprocessor Dev

MDBook Preprocessor Plugin Development

Overview

Guide for developing custom MDBook preprocessor plugins in Rust, following TDD practices.

This skill covers:

• Planning plugin functionality with the user
• Skeleton project setup and templating
• Test-driven development workflow
• Documentation (ADR, data flow, schema, I/O, examples, user stories)
• Finding existing plugins that may already solve the problem
• Creating GitHub repos in arustydev/* namespace
• Configuration and book.toml integration

This skill does NOT cover:

• Using existing preprocessors (configuring, not building)
• Writing book content
• MDBook backends/alternative renderers
• General Rust development patterns

Prerequisites

• Rust toolchain installed (rustup)
• MDBook installed (cargo install mdbook)
• GitHub CLI (gh) for repo creation
• Familiarity with Rust and Cargo

Workflow

Step 1: Discover Existing Plugins

Before building, check if a preprocessor already exists.

# Search GitHub for existing preprocessors
gh search repos mdbook-<keyword> --limit 20

# Check crates.io
cargo search mdbook-<keyword>

# Browse tagged repos
open "https://github.com/topics/mdbook-preprocessor?l=rust"

44+ preprocessors exist on GitHub. Popular ones include:

• mdbook-admonish - Material Design callouts
• mdbook-i18n - Translation support
• mdbook-cmdrun - Execute shell commands
• mdbook-mermaid - Mermaid diagrams

If existing plugin covers your use case, adopt it instead.

Step 2: Plan Plugin Functionality

Gather requirements before writing code.

2.1 Define the Problem

Ask these questions:

1. What content transformation is needed?
2. What syntax will users write in their markdown?
3. What output should be generated?
4. Which renderers should be supported? (html, pdf, epub)

2.2 Document with ADR

Create an Architecture Decision Record:

# ADR-001: <Preprocessor Name> Design

## Status
Proposed

## Context
<Why is this preprocessor needed?>

## Decision
<How will it work?>

## Consequences
<Trade-offs and implications>

2.3 Map Data Flow (Mermaid)

flowchart LR
    A[book.toml] --> B[mdbook build]
    B --> C[Load Book]
    C --> D[PreprocessorContext + Book JSON]
    D --> E[mdbook-foo stdin]
    E --> F[Transform Chapters]
    F --> G[Modified Book JSON]
    G --> H[stdout]
    H --> I[Renderer]

2.4 Define Input/Output Schema

Input (received via stdin):

[
  {
    "root": "/path/to/book",
    "config": { "book": {...}, "preprocessor": {"foo": {...}} },
    "renderer": "html",
    "mdbook_version": "0.4.40"
  },
  {
    "sections": [
      { "Chapter": { "name": "...", "content": "...", ... } }
    ]
  }
]

Output (write to stdout):

{
  "sections": [
    { "Chapter": { "name": "...", "content": "<transformed>", ... } }
  ]
}

2.5 Write User Stories

## User Stories

### US-001: Basic Usage
As a book author,
I want to write `{{#foo bar}}` in my markdown,
So that it renders as <expected output>.

### US-002: Configuration
As a book author,
I want to configure options in `book.toml`,
So that I can customize behavior without changing markdown.

### US-003: Error Handling
As a book author,
I want clear error messages when syntax is invalid,
So that I can fix issues quickly.

Step 3: Create GitHub Repository

# Create repo with standard naming
gh repo create arustydev/mdbook-<foo> \
  --public \
  --description "MDBook preprocessor for <description>" \
  --clone

cd mdbook-<foo>

# Initialize Rust project
cargo init --name mdbook-<foo>

# Apply standard templates
just apply-gist lang_rust type=bin
just apply-gist github_labels_rust
just apply-gist common

Step 4: Set Up Project Structure

mdbook-<foo>/
├── Cargo.toml
├── src/
│   ├── main.rs          # CLI entry point
│   └── lib.rs           # Preprocessor implementation
├── tests/
│   ├── integration.rs   # Integration tests
│   └── fixtures/        # Test book fixtures
├── docs/
│   ├── adr/             # Architecture decisions
│   └── book.toml        # Example configuration
├── .github/
│   └── workflows/
│       └── ci.yml       # CI/CD pipeline
└── README.md

Step 5: Configure Cargo.toml

[package]
name = "mdbook-<foo>"
version = "0.1.0"
edition = "2021"
description = "MDBook preprocessor for <description>"
license = "MIT"
repository = "https://github.com/arustydev/mdbook-<foo>"
keywords = ["mdbook", "preprocessor", "markdown"]
categories = ["command-line-utilities", "text-processing"]

[dependencies]
mdbook-preprocessor = "0.2"
mdbook-markdown = "0.2"      # For markdown parsing
serde = { version = "1", features = ["derive"] }
serde_json = "1"
anyhow = "1"
log = "0.4"
env_logger = "0.11"

[dev-dependencies]
pretty_assertions = "1"

Step 6: Implement with TDD

6.1 Write Failing Test First

// tests/integration.rs
use mdbook_foo::process_chapter;

#[test]
fn test_basic_transformation() {
    let input = r#"# Chapter
Some text with {{#foo bar}}.
"#;

    let expected = r#"# Chapter
Some text with <transformed content>.
"#;

    let result = process_chapter(input);
    assert_eq!(result, expected);
}

#[test]
fn test_no_transformation_needed() {
    let input = "# Chapter\nPlain markdown.";
    let result = process_chapter(input);
    assert_eq!(result, input);
}

6.2 Implement Preprocessor Trait

// src/lib.rs
use mdbook_preprocessor::prelude::*;
use anyhow::Result;

pub struct FooPreprocessor;

impl Preprocessor for FooPreprocessor {
    fn name(&self) -> &str {
        "foo"
    }

    fn run(&self, ctx: &PreprocessorContext, mut book: Book) -> Result<Book> {
        let config = ctx.config.get_preprocessor(self.name());

        book.for_each_mut(|item| {
            if let BookItem::Chapter(chapter) = item {
                chapter.content = process_chapter(&chapter.content);
            }
        });

        Ok(book)
    }

    fn supports_renderer(&self, renderer: &str) -> bool {
        renderer == "html"
    }
}

pub fn process_chapter(content: &str) -> String {
    // TODO: Implement transformation
    content.to_string()
}

6.3 Implement CLI Entry Point

// src/main.rs
use mdbook_preprocessor::prelude::*;
use mdbook_foo::FooPreprocessor;
use std::io;

fn main() {
    env_logger::init();

    let preprocessor = FooPreprocessor;

    if std::env::args().nth(1).as_deref() == Some("supports") {
        let renderer = std::env::args().nth(2).unwrap_or_default();
        if preprocessor.supports_renderer(&renderer) {
            std::process::exit(0);
        } else {
            std::process::exit(1);
        }
    }

    let (ctx, book) = CmdPreprocessor::parse_input(io::stdin()).unwrap();
    let result = preprocessor.run(&ctx, book).unwrap();
    serde_json::to_writer(io::stdout(), &result).unwrap();
}

6.4 Run Tests and Iterate

# Run tests (expect failures initially)
cargo test

# Implement until tests pass
cargo test -- --nocapture

# Check with clippy
cargo clippy -- -D warnings

# Format code
cargo fmt

Step 7: Test with Real Book

Create a test book:

mkdir -p tests/fixtures/test-book/src
cat > tests/fixtures/test-book/book.toml << 'EOF'
[book]
title = "Test Book"

[preprocessor.foo]
# Custom configuration here
EOF

cat > tests/fixtures/test-book/src/SUMMARY.md << 'EOF'
# Summary

- [Chapter 1](chapter1.md)
EOF

cat > tests/fixtures/test-book/src/chapter1.md << 'EOF'
# Chapter 1

Test content with {{#foo bar}}.
EOF

Test manually:

# Build and install locally
cargo install --path .

# Test with book
cd tests/fixtures/test-book
mdbook build

Step 8: Document Configuration

Add to README.md:

## Installation

```bash
cargo install mdbook-<foo>

Configuration

Add to your book.toml:

[preprocessor.foo]
option1 = "value"
option2 = true

Options

### Step 9: Set Up CI/CD

```yaml
# .github/workflows/ci.yml
name: CI

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: dtolnay/rust-toolchain@stable
      - run: cargo test
      - run: cargo clippy -- -D warnings
      - run: cargo fmt --check

  integration:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: dtolnay/rust-toolchain@stable
      - run: cargo install mdbook
      - run: cargo install --path .
      - run: cd tests/fixtures/test-book && mdbook build

Examples

Example: Simple Text Replacement

pub fn process_chapter(content: &str) -> String {
    content.replace("{{#hello}}", "Hello, World!")
}

Example: Regex-Based Transformation

use regex::Regex;

pub fn process_chapter(content: &str) -> String {
    let re = Regex::new(r"\{\{#include\s+(\S+)\}\}").unwrap();
    re.replace_all(content, |caps: &regex::Captures| {
        let path = &caps[1];
        format!("<!-- included from {} -->", path)
    }).to_string()
}

Example: Markdown Event Processing

use mdbook_markdown::{pulldown_cmark::{Event, Parser, Tag}, CMarkWriter};

pub fn process_chapter(content: &str) -> String {
    let parser = Parser::new(content);
    let events: Vec<Event> = parser.map(|event| {
        match event {
            Event::Text(text) => Event::Text(text.to_uppercase().into()),
            other => other,
        }
    }).collect();

    let mut output = String::new();
    CMarkWriter::new(&mut output).write(events.iter()).unwrap();
    output
}

Troubleshooting

Preprocessor not running

Check book.toml has [preprocessor.foo] section and binary is in PATH.

JSON parse errors

Ensure stdin/stdout handling is correct. Use env_logger for debugging:

RUST_LOG=debug mdbook build

Changes not appearing

Clear mdbook cache:

rm -rf book/
mdbook build

References

• MDBook Preprocessor Docs
• mdbook-preprocessor crate
• GitHub Topic: mdbook-preprocessor
• Example: mdbook-cmdrun