Tree Sitter Scope Extractor

Tree-Sitter AST Scope Extraction Protocol (Surgical Block Ingestion)

Overview

When an agent is debugging a stack trace pointing to a specific line number (e.g., TypeError at src/services/order.ts:142), naive agents ingest the complete 800-line source file.

Full-file ingestion:

1. Consumes 6,000+ Tokens: 90% of the file contains unrelated CRUD routes, database connection pools, and logging utilities.
2. Dilutes Debugging Attention: The LLM's attention heads must filter through 750 lines of background code to focus on the 20 lines that actually caused the bug.
3. Multiplies Turn Latency: Re-ingested on every follow-up diagnostic turn.

The Tree-Sitter AST Scope Extraction Protocol parses the file into an Abstract Syntax Tree in 2 milliseconds, extracting strictly the top-level imports and the immediate enclosing AST scope (function, method, or class block).

Full-File Dump vs. Tree-Sitter Enclosing Scope Slice

┌─────────────────────────────────────────────────────────────┐
│                 Code Context Ingestion Scope                │
│                                                             │
│  Full-File Ingestion (800 Lines / 6,500 Tokens):            │
│  • Lines 1-40: Imports & Global DB Pools                    │
│  • Lines 41-130: `createOrder`, `cancelOrder`, `listOrders` │
│  • Lines 131-180: `processOrder` <-- THE ONLY TARGET!       │
│  • Lines 181-800: 12 other unrelated accounting routines    │
│  ↳ 6,500 tokens billed, attention diluted across 800 lines  │
│                                                             │
│  Tree-Sitter Enclosing Scope (60 Lines / 420 Tokens):       │
│  • Lines 1-12: Top-level module imports (Module Contract)   │
│  • Lines 131-180: Complete `processOrder` AST block         │
│  ↳ 420 clean tokens (93.5% Cut!), 100% AST integrity!       │
└─────────────────────────────────────────────────────────────┘

The 3-Step AST Extraction Pipeline

┌───────────────────────────────────────────────────────────────────────────┐
│ 1. PARSE FILE VIA TREE-SITTER: Build Concrete Syntax Tree (2ms in C/Rust) │
│ 2. LOCATE TARGET LINE IN TREE: Find AST node spanning target line number  │
│ 3. WALK UP TO ENCLOSING SCOPE: Extract `function_declaration` node + imports│
└───────────────────────────────────────────────────────────────────────────┘

Production Python Tree-Sitter Scope Extractor

from pathlib import Path
from typing import Optional, Tuple
import tree_sitter_languages # Provides universal pre-compiled tree-sitter parsers

def extract_enclosing_ast_scope(
    file_path: Path,
    target_line: int,
    language: str = "typescript"
) -> str:
    """Extracts top-level imports and the enclosing function AST scope for target_line."""
    code_bytes = file_path.read_bytes()
    lines = code_bytes.decode("utf-8").splitlines()
    
    parser = tree_sitter_languages.get_parser(language)
    tree = parser.parse(code_bytes)
    root_node = tree.root_node

    # 1. Extract Top-Level Imports (Lines 1..N)
    import_lines = []
    for child in root_node.children:
        if "import" in child.type or "use" in child.type:
            start_row = child.start_point[0]
            end_row = child.end_point[0]
            import_lines.extend(lines[start_row:end_row + 1])

    # 2. Locate AST Node Spanning target_line (0-indexed)
    target_row = target_line - 1
    
    def find_enclosing_scope(node) -> Optional[Tuple[int, int]]:
        for child in node.children:
            if child.start_point[0] <= target_row <= child.end_point[0]:
                # If child is a function / class / method declaration, capture range
                if child.type in [
                    "function_declaration", "method_definition", 
                    "arrow_function", "class_declaration", "function_definition"
                ]:
                    return (child.start_point[0], child.end_point[0])
                # Recurse deeper into block
                deeper = find_enclosing_scope(child)
                if deeper:
                    return deeper
        return None

    scope_range = find_enclosing_scope(root_node)
    
    # 3. Assemble Output Payload
    if scope_range:
        start, end = scope_range
        scope_text = "\n".join(lines[start:end + 1])
        imports_text = "\n".join(import_lines)
        return (
            f"// --- [MODULE IMPORTS] ---\n{imports_text}\n\n"
            f"// --- [ENCLOSING AST SCOPE: Lines {start+1}-{end+1}] ---\n{scope_text}"
        )
        
    # Fallback: Return window +/- 25 lines
    start = max(0, target_row - 25)
    end = min(len(lines), target_row + 25)
    return "\n".join(lines[start:end])

Benchmark Comparison

Debugging 30 runtime exceptions across large full-stack codebases:

Agent Operational Directive

MANDATORY: When debugging specific lines or function symbols, agents must extract strictly the module imports and the enclosing Tree-sitter AST scope. Never pass entire monolithic files into context when only a single function is under investigation.