# Add Grammar

> Step-by-step guide for adding a new tree-sitter language grammar to diffsitter. Use when adding support for a new programming language.

- Skill: `afnanenayet/add-grammar` (Agent Skill)
- Install (CLI): `npx skillmds@latest add afnanenayet/add-grammar`
- Raw SKILL.md: https://api.skillmd.com/api/skills/afnanenayet/add-grammar/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: afnanenayet (https://skillmd.com/u/afnanenayet)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/afnanenayet/add-grammar

---


# Adding a New Tree-sitter Grammar to diffsitter

Follow this checklist to add support for a new language `$ARGUMENTS`. If the user did not provide a language name, ask for one before proceeding.

## Prerequisites

Read these files first:

- `build.rs` -- `grammars()` function and `GrammarCompileInfo` struct
- `src/parse.rs` -- `FILE_EXTS` map and language resolution

## Step-by-Step Checklist

### Step 1: Add the grammar as a git submodule

```sh
git submodule add https://github.com/tree-sitter/tree-sitter-$ARGUMENTS grammars/tree-sitter-$ARGUMENTS
git submodule update --init --recursive
```

Verify the submodule has a `src/` directory containing at minimum `parser.c`. Check for a `scanner.c` or `scanner.cc` file -- this determines whether you need C or C++ compilation.

### Step 2: Strip Rust bindings from the grammar repo

Tree-sitter grammar repos typically include Rust bindings that would interfere with diffsitter's custom build process. Remove them:

```sh
cd grammars/tree-sitter-$ARGUMENTS
rm -rf bindings/rust Cargo.toml build.rs
cd ../..
```

This is critical -- if `Cargo.toml` or `build.rs` remain in the grammar directory, Cargo may try to compile the grammar as a separate crate and conflict with diffsitter's build script.

### Step 3: Add `GrammarCompileInfo` to `build.rs`

In `build.rs`, add a new entry to the `grammars()` function's vector. The struct has these fields:

```rust
GrammarCompileInfo {
    /// The language name -- must match what you use in FILE_EXTS and the
    /// tree_sitter_$ARGUMENTS() constructor function name
    display_name: "$ARGUMENTS",
    /// Path to the grammar root (contains src/ directory)
    path: PathBuf::from("grammars/tree-sitter-$ARGUMENTS"),
    /// C source files in src/ to compile
    c_sources: vec!["parser.c"],           // always include parser.c
    /// C++ source files in src/ to compile (empty if no C++ scanner)
    cpp_sources: vec![],
    ..Default::default()
}
```

**Determining sources:**

- `parser.c` is always present and always compiled as C.
- If `src/scanner.c` exists, add `"scanner.c"` to `c_sources`.
- If `src/scanner.cc` exists, add `"scanner.cc"` to `cpp_sources`. Do NOT put `.cc` files in `c_sources`.
- Some grammars have no scanner at all (e.g., `json`, `go`, `java`, `c`).

**Examples from the codebase:**

C-only (parser + C scanner):
```rust
GrammarCompileInfo {
    display_name: "rust",
    path: PathBuf::from("grammars/tree-sitter-rust"),
    c_sources: vec!["parser.c", "scanner.c"],
    ..Default::default()
}
```

C + C++ scanner:
```rust
GrammarCompileInfo {
    display_name: "ruby",
    path: PathBuf::from("grammars/tree-sitter-ruby"),
    c_sources: vec!["parser.c"],
    cpp_sources: vec!["scanner.cc"],
    ..GrammarCompileInfo::default()
}
```

Parser only (no scanner):
```rust
GrammarCompileInfo {
    display_name: "json",
    path: PathBuf::from("grammars/tree-sitter-json"),
    c_sources: vec!["parser.c"],
    ..Default::default()
}
```

### Step 4: Add file extension mappings to `src/parse.rs`

Add entries to the `FILE_EXTS` static `phf_map!` in `src/parse.rs`. The key is the file extension (without dot), the value is the `display_name` from Step 3:

```rust
static FILE_EXTS: phf::Map<&'static str, &'static str> = phf_map! {
    // ... existing entries ...
    "$EXT" => "$ARGUMENTS",
};
```

Add all common extensions for the language. For example, C++ has `"cc"`, `"cpp"`, `"hpp"`, and `"tpp"`.

### Step 5: Build and verify

```sh
cargo build
```

The build script will:
1. Compile the grammar's C/C++ sources via the `cc` crate
2. Generate an `unsafe extern "C" { pub fn tree_sitter_$ARGUMENTS() -> Language; }` declaration
3. Add the language to the generated `LANGUAGES` phf_map

If the build fails, check:
- Are the source file paths correct? The build script prepends `{path}/src/` to each filename.
- Does the grammar's `src/` directory exist? Run `ls grammars/tree-sitter-$ARGUMENTS/src/`.
- Are git submodules initialized? Run `git submodule update --init --recursive`.

### Step 6: Add test data (optional but recommended)

If you want to add integration or snapshot tests, add test input files under `resources/` and write tests that parse them.

Run the full test suite:

```sh
cargo test --all
```

The `static_load_parsers` test in `src/parse.rs` will automatically test that the new grammar can be loaded by tree-sitter, since it iterates over all entries in the `LANGUAGES` map.

## Common Pitfalls

### Monorepo grammars

Some tree-sitter grammars contain multiple languages in one repository. In this case, the `path` field must point to the subdirectory containing the `src/` folder, not the repo root.

Examples from the codebase:

- **TypeScript**: The `tree-sitter-typescript` repo has `typescript/` and `tsx/` subdirectories, each with their own `src/`:
  ```rust
  GrammarCompileInfo {
      display_name: "typescript",
      path: PathBuf::from("grammars/tree-sitter-typescript/typescript"),
      // ...
  }
  GrammarCompileInfo {
      display_name: "tsx",
      path: PathBuf::from("grammars/tree-sitter-typescript/tsx"),
      // ...
  }
  ```

- **OCaml**: `tree-sitter-ocaml` has `grammars/ocaml/` subdirectory:
  ```rust
  path: PathBuf::from("grammars/tree-sitter-ocaml/grammars/ocaml"),
  ```

- **PHP**: `tree-sitter-php` has a `php/` subdirectory:
  ```rust
  path: PathBuf::from("grammars/tree-sitter-php/php"),
  ```

- **Markdown**: `tree-sitter-markdown` has a `tree-sitter-markdown/` subdirectory:
  ```rust
  path: PathBuf::from("grammars/tree-sitter-markdown/tree-sitter-markdown"),
  ```

### C++ scanner pitfalls

If the grammar has a C++ scanner (`scanner.cc`), it must go in `cpp_sources`, not `c_sources`. The build script compiles C and C++ sources with different compilers (`cc` vs `c++`) and links them into separate static libraries.

### Non-standard include paths

If the grammar's headers are not in `src/` (the default include path), use the `include_paths` field:

```rust
GrammarCompileInfo {
    display_name: "some_lang",
    path: PathBuf::from("grammars/tree-sitter-some-lang"),
    c_sources: vec!["parser.c"],
    include_paths: Some(vec![
        PathBuf::from("grammars/tree-sitter-some-lang/include"),
    ]),
    ..Default::default()
}
```

### display_name must match the constructor function

The `display_name` is used to generate the FFI symbol name `tree_sitter_{display_name}`. This must match the actual symbol exported by the compiled grammar. For languages with hyphens, use underscores in the display name (e.g., `c_sharp` for `tree-sitter-c-sharp`).

### Grammar submodule coordination

Grammar submodule updates are coordinated via the [diffsitter-grammars](https://github.com/afnanenayet/diffsitter-grammars) repository using nvchecker. For ongoing maintenance, consider adding the grammar there too.

### ABI compatibility

The grammar must have a compatible tree-sitter ABI version. The `ts_language_abi_checked` function in `src/parse.rs` verifies this at runtime, checking that the grammar's ABI version falls within `MIN_COMPATIBLE_LANGUAGE_VERSION..=LANGUAGE_VERSION`. If you see an `AbiOutOfRange` error, the grammar may need to be rebuilt with a compatible tree-sitter version.