Tool Parser Generator

Tool Parser Generator Skill

Add support for new models' tool calling formats by analyzing their chat templates and generating appropriate parser implementations for the dynamo-parsers crate (parsers/).

Parser goals (read first)

These bind every parser you add here and are the tie-breakers when vLLM, SGLang, and Dynamo disagree. The canonical list is in ../../../parsers/v2/README.md ("Parser goals"); in brief:

• Follow the model's own spec (its chat template / tool-calling guide defines the grammar), and record the spec source in the fixture YAML (a spec: URL), not just in code comments.
• Error recovery is under-specified, so a divergence from vLLM/SGLang on a recovery / edge case is expected — document it with a reason:; do not "fix" it by matching a peer.
• Never leak tool-call or reasoning markup into user-visible content/normal_text (the ↯ marker catches tool leaks).
• Recover only what is delimiter-terminated. A value followed by a delimiter (next marker, closing quote/brace/bracket) is complete -> recover it and the call, even if an outer end marker is missing; a value running to end-of-stream is ambiguous (maybe truncated mid-token) -> drop, never guess. Never invent or leak; tracing::warn! with a stable why=. A published model spec overrides this (drop if its regex requires a missing fence; cite the spec URL + quote).
• Preserve as much of the original output as possible: normal_text is the model output minus only the recognized markup spans (prefix, inter-call, and trailing text kept verbatim).
• Parsing is separate from validation — emit a call even for a tool not in the request's list; the serving layer validates.
• v1 (batch) and v2 (streaming) must always agree — same calls, same normal_text. v1 is the simple, inefficient reference (it jails/buffers the whole output, then parses); v2 parses token-incrementally (jailing only the ambiguous suffix) for lower latency. Intentional stream-vs-batch differences go in the known_divergences allowlist.

When to Use This Skill

• User asks to add tool calling support for a specific HuggingFace model
• User wants to understand how a model structures tool calls
• User needs to extend the dynamo-parsers tool_calling (or reasoning) library with new formats

Workflow

Follow this systematic workflow when the user provides a HuggingFace model name.

Phase 1: Fetch and Extract Chat Template

1. Fetch tokenizer config from HuggingFace Hub:

   URL: https://huggingface.co/{model_id}/resolve/main/tokenizer_config.json
   

2. Extract chat template:

   • Parse the JSON response
   • Look for chat_template field
   • Handle two formats:
     • String: Single template
     • Array: List of templates with name and template fields
       • Prefer tool_use template if available
       • Fall back to default template

3. Extract special tokens (if relevant):

   • bos_token, eos_token, unk_token
   • additional_special_tokens
   • Any tool-specific tokens in the config

Phase 2: Analyze Chat Template

The chat template is a Jinja template. Analyze it to identify tool call patterns:

1. Find tool-related sections:

   • Look for conditional blocks with keywords: tools, tool_call, function, available_tools
   • Extract content within {% if tools %}...{% endif %} blocks
   • Find {% for tool in tools %} loops

2. Identify markers and format:

   • Start markers: Tokens/strings before tool calls
     • Examples: <tool_call>, [TOOL_CALLS], <|python_tag|>, <｜tool▁call▁begin｜>
   • End markers: Tokens/strings after tool calls
     • Examples: </tool_call>, [/TOOL_CALLS], <｜tool▁call▁end｜>
   • Special tokens: Unicode or encoded tokens (DeepSeek, Harmony)
   • Format type:
     • JSON: Look for tojson filter, { } brackets
     • XML: Look for <function=, <parameter= patterns
     • Pythonic: Look for function(arg=val) patterns
     • DSML: Look for <｜DSML｜ tokens

3. Identify JSON structure (if JSON format):

   • Name key: Usually name or function
   • Arguments key: Usually arguments or parameters
   • Array vs single object
   • Multiple calls handling

Phase 3: Compare with Existing Parsers

Identify the family from the authoritative cheat-sheet. The full, current family-to-grammar-to-file mapping (every tool-call and reasoning family) is the "Parser families" section of ../../../parsers/v2/README.md — do not maintain a model list here, it drifts. Use it to pick the closest family, then:

1. Read that family's parser module under parsers/v1/src/tool_calling/<family>/ (the batch impl that owns the grammar) to confirm the markers and structure match the model you analyzed.
2. Review its config preset in tool_calling/config.rs (ToolCallConfig::<family>() — start/end tokens, key names, parser type) and its registry entry in tool_calling/parsers.rs (get_tool_parser_map() → ParserType).
3. Check whether a streaming (v2) parser already exists under parsers/v2/src/tool_calling/<family>.rs. New work targets v2 (pure streaming); v1 (jail-and-buffer batch) is still in use but will be removed once v2 is done.

Match the analyzed format:

• If start/end tokens and format match existing parser → Use existing parser with config
• If similar but different tokens → Adapt existing parser config
• If completely different format → Generate new parser

Phase 4: Generate or Configure Parser

Option A: Use Existing Parser (Preferred)

If a match is found, create a configuration preset:

1. Add a new preset function to parsers/v1/src/tool_calling/config.rs:

   impl ToolCallConfig {
       pub fn new_model_name() -> Self {
           Self {
               config: ParserConfig::Json(JsonParserConfig {
                   start_token: Some("<marker>".to_string()),
                   end_token: Some("</marker>".to_string()),
                   function_name_key: Some("name".to_string()),
                   function_arguments_key: Some("arguments".to_string()),
                   parser_type: JsonParserType::Basic,
               }),
           }
       }
   }
   

2. Register in parser map in parsers/v1/src/tool_calling/parsers.rs

3. Create tests to verify the configuration works

Option B: Generate New Parser (If Needed)

If no existing parser fits, generate new parser code:

1. Choose parser template based on format:

   • JSON format → Use base_json_parser.rs as template
   • XML format → Use xml/parser.rs as template
   • Custom format → Implement three core functions

2. Implement required functions:

   // Detection
   pub fn detect_tool_call_start_<name>(chunk: &str, config: &Config) -> bool
   
   // Parsing
   pub fn try_tool_call_parse_<name>(
       message: &str,
       config: &Config,
       tools: Option<&[ToolDefinition]>,
   ) -> Result<(Vec<ToolCallResponse>, Option<String>)>
   
   // End detection (for streaming)
   pub fn find_tool_call_end_position_<name>(chunk: &str, config: &Config) -> usize
   

3. Use regex for token matching:

   • Use OnceLock<Regex> for compiled regexes
   • Escape special characters properly
   • Handle partial tokens for streaming

4. Parse JSON/XML content:

   • Use serde_json for JSON parsing
   • Use regex for XML extraction (or XML parser if complex)
   • Build ToolCallResponse structs

5. Add to appropriate directory:

   • JSON variants → json/ directory
   • XML variants → xml/ directory
   • New format → Create new subdirectory

Phase 5: Generate Tests

For any new parser or configuration, generate comprehensive tests:

1. Basic tests:

   • Detection of start markers
   • Parsing single tool call
   • Parsing multiple tool calls
   • Normal text extraction

2. Edge cases:

   • Empty arguments
   • Missing fields
   • Malformed JSON/XML
   • Partial tokens (streaming)

3. Integration tests:

   • End-to-end with real model outputs (if available)
   • Tool validation (if tools list provided)

4. Add tests to appropriate location:

   • Inline in parser file (in #[cfg(test)] module)
   • Or in parsers/v1/src/tool_calling/tests.rs

Phase 6: Integration

1. Update module exports:

   • Add mod declaration in parent mod.rs
   • Export functions as needed

2. Register parser in parsers.rs if new parser:

   • Add to get_tool_parser_map() function
   • CRITICAL: Update test_get_available_tool_parsers() test
   • Add your new parser name to the available_parsers array in the test

3. Document the parser:

   • Add doc comments explaining format
   • Include example input/output
   • Reference model family

4. Run tests (from the repo root):

   cargo test -p dynamo-parsers tool_calling
   

5. Verify:

   • Test with actual model output if possible
   • Verify streaming behavior
   • Check error handling

Key Reference Files

This repo (dynamo-parsers):

• parsers/v1/src/tool_calling/ - All tool call parsers
• parsers/v1/src/tool_calling/config.rs - Configuration presets
• parsers/v1/src/tool_calling/parsers.rs - Parser registry
• parsers/v1/src/reasoning/ - Reasoning parsers (same workflow if you're adding a <think>-style reasoning parser instead of a tool parser)

Note: chat-template loading/structures (tokcfg.rs, template.rs) live upstream in ai-dynamo/dynamo's lib/llm, not in this crate. Here you fetch the model's tokenizer_config.json directly from HuggingFace (Phase 1) — you don't need the dynamo template code to build a parser.

Reference Implementations:

• sglang: https://github.com/sgl-project/sglang/tree/main/python/sglang/srt/function_call
  • Look at detector pattern (base_format_detector.py)
  • Model-specific detectors (qwen25_detector.py, deepseekv3_detector.py, etc.)
• vLLM: https://github.com/vllm-project/vllm/tree/main/vllm/tool_parsers
  • Look at abstract_tool_parser.py
  • Model-specific parsers (llama_tool_parser.py, qwen3xml_tool_parser.py, etc.)
• HuggingFace: https://huggingface.co/docs/transformers/chat_templating

Example: Adding Support for a New Model

User: "Add tool calling support for Qwen/Qwen2.5-72B-Instruct"

Step 1: Fetch tokenizer config

• Use WebFetch to get https://huggingface.co/Qwen/Qwen2.5-72B-Instruct/resolve/main/tokenizer_config.json

Step 2: Analyze chat template

• Extract chat_template field
• Identify {% if tools %} block
• Find markers: Likely <tool_call> and </tool_call>
• Identify format: Check for JSON with tojson filter

Step 3: Compare with existing parsers

• Read parsers/v1/src/tool_calling/config.rs
• Check ToolCallConfig::hermes() - uses <tool_call> markers
• Check if Qwen format matches hermes format

Step 4: Use or adapt existing parser

• If matches hermes: Create qwen2_5() config preset
• If different: Generate new parser or adapt base_json_parser

Step 5: Generate tests

• Create test cases with example Qwen tool calls
• Test detection, parsing, and edge cases

Step 6: Integrate

• Add config preset to config.rs
• Register in parser map (get_tool_parser_map())
• Update test_get_available_tool_parsers() test
• Run tests
• Document

Tips

• Always prefer existing parsers: Most models can use existing parsers with different configs
• Read reference implementations: sglang and vLLM often have parsers for popular models
• Use WebFetch for HF models: Don't assume - always fetch actual tokenizer config
• Test with real outputs: If possible, get actual model outputs to test against
• Keep it simple: Prefer straightforward regex over complex parsing when possible
• Document well: Future you (or others) will thank you

Common Patterns

The grammar-recognition guide — how to spot each pattern in a chat template — is in references/parser-patterns.md. The authoritative family-to-grammar-to-file mapping is the "Parser families" cheat-sheet in ../../../parsers/v2/README.md. Use those instead of re-listing patterns here.

Minimal Changes Philosophy

1. First: Try existing parser with new config
2. Second: Adapt existing parser with minor tweaks
3. Last resort: Create entirely new parser

Most models (>80%) can use existing parsers with appropriate configuration.