# Marmite Development > Guidelines and workflows for contributing to the marmite codebase - covers code quality, testing, architecture patterns, and contribution checklists - Skill: `rochacbruno/marmite-development` (Agent Skill) - Install (CLI): `npx skillmds@latest add rochacbruno/marmite-development` - Raw SKILL.md: https://api.skillmd.com/api/skills/rochacbruno/marmite-development/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: rochacbruno (https://skillmd.com/u/rochacbruno) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/rochacbruno/marmite-development --- # Marmite Development Guide This skill is for developers contributing to the marmite codebase. For building sites with marmite, use the `marmite` skill instead. - Repository: https://github.com/rochacbruno/marmite - Site: https://marmite.blog - License: AGPL-3.0-or-later - Language: Rust (edition 2021) ## Tools to call during development ### Show urls ``` marmite input_folder --show-urls ``` ``` { "archives": [ "/archive-2027.html", "/archive.html" ], "authors": [ "/author-jao.html", "/authors.html" ], "feeds": [ "/index.rss", "/internet.rss", "/tag-python.rss", "/author-jao.rss", "/archive-2026.rss" ], "file_mappings": [], "misc": [ "/index.html" ], "pages": [ "/about.html", "/pages.html" ], "pagination": [ "/pages-1.html", "/tag-dev-1.html", "/tag-python-1.html", "/author-jao-1.html", "/archive-2026-1.html", "/index-1.html", ], "posts": [ "/code-django.html", "/code-python.html", ], "redirects": [], "series": [], "streams": [ "/code.html", "/streams.html" ], "summary": { "archives": 1, "authors": 1, "feeds": 5, "file_mappings": 0, "meta": { "absolute_urls": false, "url": "" }, "misc": 1, "pages": 1, "pagination": 1, "posts": 2, "redirects": 0, "series": 0, "streams": 2, "tags": 1, }, "tags": [ "/tag-python.html", "/tags.html" ] } ``` This tool allows the agent to: - compare urls before and after a refactor - check generated content slug - count number of contents ### init site ``` marmite input_folder --init-site ``` Initialize a new site structure from scaffolding. ### New post ``` marmite input_folder --new "title" -t tag ``` Create a new post. ### More Use `marmite --help` to check what else is available. ## Code Quality Requirements ### Formatting and Linting Every change must pass formatting and linting before it can be committed. ```bash # Format code (always run first) mask fmt # Check formatting + clippy lints mask check # Pedantic clippy (run before opening a PR) mask pedantic ``` `mask check` runs `cargo fmt -- --check` followed by `cargo clippy`. Both must pass clean. `mask pedantic` adds `-W clippy::pedantic` for stricter analysis - fix pedantic issues only after confirming with the project maintainer. Do not suppress clippy warnings with `#[allow(...)]` unless there is a justified reason. If a suppression is needed, use the narrowest possible scope (on the item, not the module). ### Safe Rust Practices - **No `unwrap()` in non-test code.** Use `Result` and `Option` with proper error propagation (`?` operator), `unwrap_or`, `unwrap_or_else`, `unwrap_or_default`, or pattern matching. Reserve `unwrap()` for test code and cases where the value is guaranteed at compile time (e.g., embedded assets loaded via `rust_embed`). - **No `panic!` in library code.** Return `Result` or `Option` instead. Use `error!` from the `log` crate to report problems, then return an error or a sensible default. - **No unsafe code** unless absolutely necessary and approved by the maintainer. - **Handle all `Result` types.** Do not silently discard errors with `let _ = ...` unless the discard is intentional and documented. - **Prefer `&str` over `String`** in function parameters when ownership is not needed. - **Use `Arc` for shared ownership** across threads. The project uses `Arc` and `Arc` for passing shared state. - **Avoid `.clone()` when a borrow suffices.** Cloning is acceptable when the borrow checker requires it or when the data needs to outlive the current scope, but do not clone as a reflex. - **Use `LazyLock` for static initialization** instead of `lazy_static!` in new code. The project is transitioning to `std::sync::LazyLock`. ### Error Handling Patterns The project uses the `log` crate for user-facing messages: ```rust use log::{info, warn, error, debug, trace}; // User-facing progress info!("Processing {} files", count); // Recoverable problems the user should know about warn!("Skipping file with invalid frontmatter: {}", path.display()); // Fatal or important errors error!("Failed to write output: {}", err); // Developer debugging (visible with -vv) debug!("Resolved slug: {}", slug); // Detailed tracing (visible with -vvv) trace!("Template context: {:?}", context); ``` Functions that can fail should return `Result` or `Option`. Use `process::exit(1)` only in `main.rs` for top-level CLI errors. ### Naming Conventions - `snake_case` for functions, methods, variables, and modules - `PascalCase` for types, structs, enums, and traits - `SCREAMING_SNAKE_CASE` for constants and static variables - CLI flags: use `#[arg(long)]` with descriptive names (e.g., `--skip-image-resize`, `--init-templates`). Clap automatically converts underscores to hyphens. - Config fields: use `snake_case` matching the YAML key name. Always add `#[serde(default)]` with a default function. ## Testing Every change needs tests. The project has two testing layers: ### Unit Tests (`src/tests/`) Unit tests call marmite code directly. Each module has a corresponding test file. ``` src/tests/ content.rs # Content parsing, slug generation, streams shortcodes.rs # Shortcode regex matching and expansion parser.rs # Markdown to HTML conversion tera_functions.rs # Custom Tera template functions tera_filter.rs # Custom Tera filters feed.rs # RSS feed generation gallery.rs # Image gallery processing site.rs # Site generation logic templates.rs # Template initialization embedded.rs # Embedded asset loading server.rs # HTTP server image_resize.rs # Image resizing image_provider.rs # Image download theme_manager.rs # Theme installation ``` Unit test pattern: ```rust use super::*; use crate::config::Marmite; #[test] fn test_descriptive_name_of_what_is_tested() { // Setup let config = Marmite::default(); // Act let result = function_under_test(&config); // Assert assert_eq!(result, expected_value); } ``` Run unit tests: ```bash mask test_unit # or a specific test cargo test --bin marmite test_name ``` ### Integration Tests (`tests/`) Integration tests run marmite as a subprocess via `std::process::Command` and assert on the output files. They test end-to-end behavior. ``` tests/ basic_functionality.rs # CLI help, version, minimal site generation content_generation.rs # Content processing end-to-end features.rs # Feature-specific integration tests streams.rs # Stream content type wikilinks_integration.rs # Wikilink resolution highlight.rs # Syntax highlighting image_resize_integration.rs # Image resizing pipeline themes.rs # Theme loading and application ``` Integration test pattern: ```rust use std::fs; use std::process::Command; use tempfile::TempDir; #[test] fn test_feature_name_end_to_end() { let temp_dir = TempDir::new().unwrap(); let input_dir = temp_dir.path().join("input"); let output_dir = temp_dir.path().join("output"); fs::create_dir_all(input_dir.join("content")).unwrap(); fs::write( input_dir.join("marmite.yaml"), "name: Test Site\ntagline: Test", ).unwrap(); fs::write( input_dir.join("content").join("test.md"), "# Test Page\n\nContent here.", ).unwrap(); let output = Command::new("cargo") .args(["run", "--quiet", "--", input_dir.to_str().unwrap(), output_dir.to_str().unwrap()]) .output() .expect("Failed to execute marmite"); assert!(output.status.success()); // Assert on generated files let html = fs::read_to_string(output_dir.join("test.html")).unwrap(); assert!(html.contains("Test Page")); } ``` Run integration tests: ```bash mask test_integration # or a specific file cargo test --test basic_functionality ``` Run all tests: ```bash mask test ``` ### What to Test - **New features:** Both unit tests for the logic and integration tests for CLI/end-to-end behavior. - **Bug fixes:** A test that reproduces the bug and verifies the fix. The test should fail without the fix applied. - **Config options:** Test default values, custom values, and edge cases. - **CLI flags:** Integration test that passes the flag and checks the result. ## Avoiding Breaking Changes Marmite is used by people who depend on its current behavior. Follow these principles: - **Prefer feature flags over behavior changes.** If a change alters existing behavior, add a config option to opt in. Make the current behavior the default so existing sites are unaffected. - **New config fields must have sensible defaults.** Use `#[serde(default = "default_function")]` so that existing `marmite.yaml` files without the new field continue to work. - **Do not remove or rename existing config fields.** If a field needs to be superseded, deprecate it (log a warning when it is used) and support both the old and new field. - **Do not remove or rename existing CLI flags.** Use `#[arg(hide = true)]` to hide deprecated flags from `--help` while still accepting them. - **Do not change the default output structure** (file names, directory layout) without a migration path or a feature flag. - **Template variable changes must be backward-compatible.** Adding new variables is fine; removing or renaming existing ones breaks custom templates. ## Documentation Requirements Every new feature must be documented in multiple places. The marmite website is generated from `example/content/`, so documentation and the site are the same thing. ### Blog post for new features Create a new markdown file in `example/content/` documenting the feature. Follow existing patterns: - Use a dated filename: `YYYY-MM-DD-feature-name.md` - Include frontmatter with title, description, and relevant tags - Write a practical guide showing how to use the feature, with examples - If the feature adds CLI flags, also update `example/content/2024-11-26-marmite-command-line-interface.md` ### LLM documentation (`example/ai/llms.txt`) Add an entry to `example/ai/llms.txt` under the appropriate section for any significant change: new CLI flag, new config option, new content capability, new template variable, or new shortcode. This file is served at `marmite.blog/llms.txt` and is used by AI agents to understand marmite's capabilities. ### Agent skill references (`.agents/skills/marmite/references/`) Update the corresponding reference file when the feature changes: | What changed | File to update | |-------------|---------------| | Config options | `references/config-reference.md` | | CLI flags | `references/cli-reference.md` | | Frontmatter fields | `references/frontmatter.md` | | Template variables or functions | `references/tera-templates.md` | | Shortcodes | `references/shortcodes.md` | | Content organization (streams, series, fragments) | `references/content-organization.md` | | Deployment or hosting | `references/deployment-guide.md` | | Comment systems | `references/comment-system.md` | These files are compiled into the binary via `rust_embed` and installed with `--skill-install`. They must stay accurate. When adding a new feature, add to the top level SKILL.md ### Release notes Every new feature and bug fix must be added to the current draft release notes. Find the latest release notes file with `stream: draft` in frontmatter under `example/content/release-notes/`. The filename pattern is `marmite-X-Y-Z-release-notes.md`. - For new features, add a section describing what the feature does and how to use it. - For bug fixes, describe what was broken and how it is fixed. - If no draft release notes file exists, create one following the same pattern, using the next expected version number. ## Template Changes Marmite has two sets of templates that must be kept in sync: ### Main theme (`example/templates/` and `example/static/`) This is the default theme used by marmite when no custom theme is set. Changes to templates, CSS, JavaScript, or static assets go here first. - `example/templates/` - Tera HTML templates (`base.html`, `content.html`, `list.html`, `group.html`) - `example/static/` - CSS, JavaScript, fonts, colorschemes ### Alternative theme - `theme_template` (`example/theme_template/`) This is the scaffolding theme used when users run `marmite --start-theme `. It provides a starting point for new custom themes. It has its own copies of templates and static assets: - `example/theme_template/templates/` - Template files - `example/theme_template/static/` - Static assets (CSS, JS, fonts, colorschemes) **When you change any template or static asset in the main theme, you must also apply the corresponding change to `theme_template`.** The two themes can differ in styling and layout details, but they must both support the same template variables, blocks, and structural features. If a new template block or variable is added in the main theme, `theme_template` must also include it so that users who scaffold a new theme get a working starting point. After making template changes, verify both themes work: ```bash # Test with the main theme mask serve # Test with theme_template mask serve_theme ``` ## Adding New Features ### Every config parameter needs a CLI argument When you add a new field to `marmite.yaml`, it must also be settable via a CLI flag. This allows users to override config without editing files, and is essential for CI/CD and scripting. The full process: 1. **Add the config field** in `config.rs`: ```rust #[serde(default = "default_my_feature")] pub my_feature: bool, ``` Add the default function: ```rust fn default_my_feature() -> bool { false } ``` 2. **Add the CLI flag** in `cli.rs` inside the `Configuration` struct: ```rust /// Enable my feature #[arg(long)] pub my_feature: bool, ``` 3. **Wire the override** in `config.rs` inside `Marmite::override_from_cli_args()`: ```rust if cli_args.configuration.my_feature { self.my_feature = true; } ``` 4. **Document it:** - Add a commented-out example in `example/marmite.yaml` - Update `.agents/skills/marmite/references/config-reference.md` - Update `example/content/2024-11-26-marmite-command-line-interface.md` if it is a CLI flag ### Adding a new CLI command 1. Add the flag to `Cli` in `cli.rs` with `#[arg(long)]` and a doc comment (becomes `--help` text). 2. Add the handler in `run_cli()` in `main.rs` at the correct position in the command chain. 3. If the command does not need `input_folder`, handle it before `input_folder` resolution. 4. Update `determine_verbosity()` if the command should auto-set verbosity. 5. Update the CLI docs. ### Adding a new template function 1. Implement the function struct in `tera_functions.rs` (implement `tera::Function`). 2. Register it in `site.rs` where other functions are registered (search for `tera.register_function`). 3. Update `.agents/skills/marmite/references/tera-templates.md`. 4. Add the function name to `write_template_context_files` in `site.rs` (the `functions` vec). 5. Add it to the `TERA_BUILTIN_FILTERS` or function completion list in `assets/toolbar/editor.js`. ### Adding a new template filter 1. Implement the filter function in `tera_filter.rs` (implement `tera::Filter`). 2. Register it in `site.rs` where other filters are registered (search for `tera.register_filter`). 3. Update `.agents/skills/marmite/references/tera-templates.md`. 4. Add the filter name to `write_template_context_files` in `site.rs` (the `filters` vec). 5. Add it to the editor's filter completion list in `assets/toolbar/editor.js`. ### Adding a new template variable 1. Insert the variable into the Tera context in `site.rs` (search for `context.insert`). 2. Determine which templates receive it: - Global context (`render_templates` function) - available to all templates - Content context (`handle_content_pages` function) - available to `content.html` - List context (`render_list_page` function) - available to `list.html` - Group context (`render_group_page` function) - available to `group.html` 3. Update `.agents/skills/marmite/references/tera-templates.md`. 4. Add the variable to the relevant template list in `write_template_context_files` in `site.rs`. ### Template rendering architecture Marmite uses Tera 2.0. Key points for contributors: - **Template loading** (`initialize_tera` in `site.rs`): loads embedded default templates first, then user templates from `templates_path` which override the defaults. The `preprocess_template` function in `embedded.rs` auto-converts Tera 1.x syntax (dot-notation indexing, positional test args, `ignore missing` includes) to Tera 2.0 syntax. - **Template context** is built in layers: global context (site config, fragments, menu) is shared by all templates, then each template type adds its own variables. - **Fragment rendering**: files prefixed with `_` in the content directory (e.g., `_hero.md`, `_sidebar.md`) are rendered through Tera with the global context, then converted to HTML, and injected as variables into all templates. - **Template fallback**: list pages use a `custom_{filename},list.html` pattern - Tera tries `custom_index.html` first, falls back to `list.html`. - **Error handling during --serve**: template rendering errors are logged but do not crash the server. The previous good build remains served. - **Template context files**: during `--serve`, `template.{name}.context.json` files are written to the output folder listing available variables, functions, and filters per template type. These are consumed by the editor for autocomplete. ### Tera 2.0 notes for contributors Marmite uses Tera 2.0. When writing or modifying templates: - **No macros**: macros were removed in Tera 2.0. Shortcodes use their own `{% shortcode name() %}` syntax which is processed by marmite's shortcode processor before Tera. - **Array indexing**: use `items[0]` not `items.0` (the preprocessor auto-converts old syntax). - **Optional chaining**: `{{ content?.extra?.key }}` is safe for deeply nested access. - **Renamed filters**: `escape` is now `escape_html`, `linebreaksbr` is `newlines_to_br`, `as_str` is `str`. Marmite provides compatibility shims for the old names. - **Removed filters**: `addslashes`, `spaceless`, `get_env`, `concat` are gone from Tera. `striptags`, `slice`, `trim_start_matches`, and `date` are provided by marmite as custom filters. - **Tests use keyword args**: `value is divisible_by(n=3)` not `value is divisibleby(3)`. ### Adding embedded assets New embedded assets follow the `rust_embed` pattern in `embedded.rs`: ```rust #[derive(Embed, Debug)] #[folder = "$CARGO_MANIFEST_DIR/path/to/folder/"] pub struct MyAssets; pub static EMBEDDED_MY_ASSETS: LazyLock)>> = LazyLock::new(|| { let mut files: Vec<(String, Vec)> = Vec::new(); for name in MyAssets::iter() { let file = MyAssets::get(name.as_ref()) .expect("Failed to get embedded asset - this is a build-time error"); files.push((name.clone().to_string(), file.data.clone().to_vec())); } files }); ``` ## Dependency Management - **Minimize new dependencies.** Marmite ships as a single binary. Every dependency adds compile time and binary size. Prefer the standard library when possible. - **Pin major versions** in `Cargo.toml` (e.g., `serde = "1.0"`). Dependabot handles patch and minor updates. - **Check for security advisories** with `cargo audit` before adding new crates. - **Feature-gate heavy optional dependencies.** If a feature requires a large crate, consider making it optional behind a Cargo feature flag. ## Project Architecture ### Content processing pipeline 1. Walk the content directory (`walkdir`) 2. For each `.md` file, call `Content::from_markdown()`: - Extract frontmatter (`frontmatter_gen` - supports YAML/TOML/JSON) - Determine post vs page (date presence) - Detect stream from frontmatter or filename prefix - Generate slug from frontmatter, title, or filename - Convert markdown to HTML via comrak - Post-process the HTML (fix internal links, replace `@/` media refs) - Process shortcodes if enabled 3. Build taxonomy indexes (tags, authors, archive, streams, series) 4. Resolve backlinks and related content 5. Render Tera templates with content and site data 6. Write HTML output, copy static/media, resize images ### HTML post-processing over raw markdown rewriting When a feature needs to rewrite URLs, paths, or other content in the output, always operate on the rendered HTML rather than on the raw markdown before comrak processes it. Comrak turns markdown into HTML and applies its own transformations (wrapping images in `
`, generating anchor links, rendering code fences into `` blocks, etc.). A naive find-and-replace on the raw markdown cannot distinguish between content the user intends to render (an image `src`, a link `href`) and content that should be left literal (code blocks, inline code, plain prose). By the time HTML is produced, that distinction is already made - real attributes live in tags, and literal text is inside `` or plain text nodes. Existing functions that follow this pattern: - `fix_internal_links()` in `parser.rs` - rewrites `` attributes to turn `.md` references into `.html` slugs. Runs at the end of `get_html_with_options()`. - `fix_wikilinks()` in `parser.rs` - rewrites `` attributes to resolve wikilink targets. Called in `site.rs` after content is collected. - `fix_at_media_refs()` in `content.rs` - rewrites `src="@/..."` and `href="@/..."` to `src="media/{slug}/..."`. Runs after `get_html_with_options()` returns in `from_markdown()`. When implementing a new feature that needs to transform paths, URLs, or text in content output, follow the same approach: 1. Let comrak convert markdown to HTML first. 2. Write a regex that targets the specific HTML attribute or tag (e.g., `src="..."`, `href="..."`, ``). 3. Define the regex pattern as a constant in `re.rs` with a doc comment. 4. Apply the replacement on the HTML string after `get_html_with_options()` returns. 5. Test that the replacement does not affect code blocks or plain text containing the same pattern. ### CLI command routing `main()` parses args with `cli::Cli::parse()`, then `run_cli()` checks flags in priority order. Each handler returns early after completing its task. Commands that do not need `input_folder` are handled before the folder resolution. ### Key modules | Module | Responsibility | |--------|---------------| | `cli.rs` | Clap argument definitions | | `config.rs` | `Marmite` struct, YAML deserialization, CLI overrides | | `content.rs` | `Content` struct, frontmatter parsing, slug generation | | `site.rs` | Core site generation (~2000 lines), template rendering, taxonomy building | | `templates.rs` | Template initialization, theme loading | | `tera_functions.rs` | Custom Tera functions (`url_for`, `group`, `get_posts`, etc.) | | `tera_filter.rs` | Custom Tera filters (`default_date_format`, `remove_draft`) | | `shortcodes.rs` | Shortcode processing with regex pattern matching | | `parser.rs` | Markdown to HTML conversion with comrak | | `feed.rs` | RSS feed generation | | `embedded.rs` | Embedded assets via `rust_embed` | | `server.rs` | Built-in HTTP server with WebSocket live reload | | `image_resize.rs` | Parallel image resizing with rayon | ## Contribution Checklist ### After any code change 1. `mask fmt` 2. `mask check` 3. `mask test` ### After implementing a new feature 1. Format and lint: `mask fmt` and `mask check` 2. Add tests: unit tests in `src/tests/` for module logic, integration tests in `tests/` for CLI/end-to-end behavior 3. Write a new blog post in `example/content/` documenting the feature (see "Documentation Requirements") 4. Update CLI docs in `example/content/2024-11-26-marmite-command-line-interface.md` if CLI flags changed 5. Add an entry to `example/ai/llms.txt` under the appropriate section 6. Update the draft release notes - find the file with `stream: draft` in `example/content/` and add the feature 7. Update the relevant reference files in `.agents/skills/marmite/references/` (see table in "Documentation Requirements") 8. If templates or static assets changed, apply the same changes to `example/theme_template/` (see "Template Changes") ### After implementing a bug fix 1. `mask fmt` and `mask check` 2. Add or update tests that cover the fixed behavior 3. Update the draft release notes - find the file with `stream: draft` in `example/content/` and describe what was broken and how it is fixed ### Before opening a PR 1. `mask fmt` and `mask check` - must pass clean 2. `mask test` - all tests must pass 3. `mask pedantic` - run and discuss pedantic issues with the maintainer before fixing 4. Verify the example site builds correctly: `mask serve`, check in a browser 5. If templates changed, also verify with `mask serve_theme` to test the `theme_template` variant 6. Ensure documentation is complete: blog post, llms.txt entry, skill references, draft release notes 7. Ensure no unrelated changes are staged ## Development Environment ### First-time setup ```bash # Install mask task runner cargo install mask # Install pre-commit hook (runs mask pedantic before every commit) mask install_hook # Build and serve the example site with live reload mask serve ``` The site runs at http://localhost:8000 with auto-rebuild on file changes. **Pre-commit hook:** The hook runs `mask pedantic` (format check + clippy with pedantic warnings) before every commit. If a check fails, the commit is blocked. To bypass for work-in-progress commits, use `git commit --no-verify`. **Agent note:** Before starting development work, verify the hook is in place. Check if `.git/hooks/pre-commit` exists and contains `mask pedantic`. If not, run `mask install_hook`. ### Useful commands | Command | What it does | |---------|-------------| | `mask serve` | Build and serve example site with live reload and trace logging | | `mask watch` | Watch for changes, rebuild without serving | | `mask fmt` | Format code with `cargo fmt` | | `mask check` | Check formatting + run clippy | | `mask test` | Run all tests (unit + integration) | | `mask test_unit` | Run unit tests only | | `mask test_integration` | Run integration tests only | | `mask build` | Build release binary | | `mask pedantic` | Run clippy with pedantic warnings | | `mask install_hook` | Install pre-commit hook that runs `mask pedantic` | | `mask fix` | Auto-apply clippy fixes | | `mask serve_theme` | Serve with the `theme_template` theme | | `mask serve_site` | Serve the actual marmite.blog site locally | | `mask coverage_llvm` | Generate HTML coverage report (requires `cargo-llvm-cov`) | ### Running specific tests ```bash # A specific unit test by name cargo test --bin marmite test_name_here # A specific integration test file cargo test --test basic_functionality # All tests with output cargo test -- --nocapture ```