# Recursive Language Model > Use when a developer wants a small or local model to take over an agentic task a frontier model one-shots — "can a small model do this whole case", "the prompt is too big for the small model", "hill-climb the local model on this task". Decomposes the task into bounded, flat-context steps behind the same call contract; includes the training path. - Skill: `understudylabs/recursive-language-model` (Agent Skill, multi-file: 2 files) - Install (CLI): `npx skillmds@latest add understudylabs/recursive-language-model` - Raw SKILL.md: https://api.skillmd.com/api/skills/understudylabs/recursive-language-model/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - Author: understudylabs (https://skillmd.com/u/understudylabs) - Updated: 2026-09-22 - Page: https://skillmd.com/skills/understudylabs/recursive-language-model --- # Recursive language model (decomposition harness) A methodology, not a library — the coding agent writes the loop to fit the workload. Use it when a frontier model does a task in **one giant agentic prompt** (many tool declarations + a context that compounds as raw tool results pile up) that a smaller local model cannot. Instead of shrinking the model, change the *harness*: have the small model solve the task as a sequence of small, bounded steps, each with a deliberately flat context. Crucially, the RLM wraps the small model **behind the incumbent's existing call contract** — the workload's callers send the same request and get the same answer; the decomposition happens *inside*. That makes the specialist a **drop-in replacement** for the frontier generalist, not a rewrite of the system around it. This composes two other skills: it needs [`../understand-workload/SKILL.md`](../understand-workload/SKILL.md) to know the task, its steps, and the success criteria, and [`../design-simulated-environment/SKILL.md`](../design-simulated-environment/SKILL.md) so the small model has a real, scorable place to run the whole case. ## The two things that make the giant prompt fail for a small model - **Fixed tool overhead.** The whole tool catalog is re-sent every turn, often the large majority of the prompt, for a turn that uses one tool. Fix: **per-step tool subsetting** — show the model only the one or few tools that step needs. - **Compounding context.** Each raw tool result (some very large) is appended and re-sent on every later turn, so the same bytes are billed many times and the window fills. Fix: **summarize each result** into the few facts the task needs and carry only the summary forward — context stays small and roughly flat. ## Recipe 1. **Frame the loop.** State = the task + a short scratchpad of notes + the step number. Each step, give the model only: the task, the relevant tool subset, and the current scratchpad. Ask for exactly one next action (or "done"). 2. **Subset the tools per step** from the workload's tool catalog — never the full surface. Pick by intent (the step's verb → its tool class). 3. **Run the action** against the simulated environment (so there are no live side effects and any tool call gets a result), then **summarize** the result into the scratchpad. The raw result must not re-enter the next prompt — summarizing well is where recall is won or lost; extract exactly the fields the gold needs. 4. **Recurse when a step is itself big.** A step may be "spawn a sub-loop on sub-task X" (mirroring the workload's own sub-agents) — a fresh small-context loop whose summary returns to the parent. This is the "recursive" in RLM. 5. **Stop** when the model writes its answer/observations or hits a step budget; score the final state with the environment's validator. ## Pattern: structured intermediate artifacts When the task is long-horizon synthesis (drafting, analysis, multi-document work) rather than tool routing, have the loop emit **explicit intermediate artifacts** — a structured plan, an extracted-facts table, a checklist the deliverable must satisfy — and make later steps consume only those artifacts, not free-form scratchpad prose. Each artifact is checkable (schema, required keys) at the step that produces it, so errors surface mid-loop instead of in the final deliverable. The gap this closes is large: measured on an internal long-horizon benchmark, 2026-06-03, a structured-artifact workflow scored 57/59 where a raw free-form RLM loop scored 3/59 with the same models. ## The orchestrator and the runner need not be the same model The hardest sub-skill is the orchestration — planning the decomposition and authoring good sub-steps. The high-*volume* work is the bounded execution (run the step, summarize the result). These can run on **different models**: a capable model orchestrates (low call count, where quality matters) while a **small local model runs the bounded steps** (high call count, where cost lives). This "big orchestrator + small runner" split keeps planning quality high and displaces the fan-out — usually a better cost shape than forcing one small model to do both. Decide per band: if the small model's *orchestration* is the gap (bad plans, no grounding), keep a capable orchestrator and make the small model the runner; if only *execution* is the gap, the small model can be both under a tighter scaffold. ## What to measure (what's possible) **Outcome:** - **Decomposition factor** — small-model steps ÷ the teacher's steps to reach the same scored final state. 1 = one-shot parity; higher = more bounded passes per teacher call. - **Recall / precision / policy** vs the frontier (from the env validator), and how they move as you improve summarization, tool subsetting, and step prompts — this is the hill-climbing loop. - **Flat-context win** — peak context vs the teacher's compounding context, and the resulting cost/latency at parity. **Process (where it breaks — diagnose, don't just score).** A small model that scores near-zero usually is *not* failing to know it should decompose (the harness prompts that); it decomposes *badly*. Outcome alone can't tell you which sub-skill to fix, so instrument the trajectory: - **grounding rate** — fraction of steps that actually read the prior result vs reasoning in a vacuum (the dominant small-model failure: it stops reading its own output). - **think-vs-act ratio** — output spent deliberating vs taking the next action. - **compaction** — does the scratchpad/working context stay bounded, or balloon turn over turn? - **repetition** — does it loop / re-derive the same step? - **sub-summary accuracy** — are the summaries it carries forward actually correct? These pinpoint *which* band to fix — plan vs execute vs ground vs combine — and they double as **dense per-step rewards** when you train (reward grounding/compaction/correct summaries, not just the final pass/fail). Empirically, scaling the base model buys *grounding and execution* but not *less over-thinking* — so part of any gap is capacity, part is scaffold; the process profile tells you which. Report the local model's reachable score, the decomposition factor, the process profile, and the remaining gap — feeding the route decision ([`../run-local-model-lab/SKILL.md`](../run-local-model-lab/SKILL.md)) and, only if local rungs genuinely can't close it, [`../prepare-verifier-handoff/SKILL.md`](../prepare-verifier-handoff/SKILL.md). ## Training Path When the RLM loop itself is the thing that must improve, follow [`references/pedagogical-training.md`](references/pedagogical-training.md). It turns the decomposition harness into a verifiers-style training surface, separates privileged training context (`c`) from deploy-time input (`x`), measures surprise concentration, explains how to read a live GRPO training signal, and decides whether the next rung is local pedagogical SFT, on-policy repair, pedagogical RL, or a hosted verifier handoff. That reference also covers **harvesting cold-start SFT data from the big-orchestrator/small-runner split** — collect *good* trajectories from a strong root, don't SFT on a weak model's failures. ## Safety Gates - Run only against the simulated environment or live tools the user approved — never fire real side effects while exploring. - Keep captured customer data local; summaries and fixtures must be synthetic before anything is shared or committed. ## References - [`references/pedagogical-training.md`](references/pedagogical-training.md) — train the RLM policy (verifiers shape, baselines, concentration, training arms, reading the training signal). - [`../understand-workload/SKILL.md`](../understand-workload/SKILL.md) - [`../design-simulated-environment/SKILL.md`](../design-simulated-environment/SKILL.md) - [`../run-local-model-lab/SKILL.md`](../run-local-model-lab/SKILL.md) - [`../ladder/SKILL.md`](../ladder/SKILL.md) — vibe-check the hill-climbed local model vs the frontier.