Tool Budget Audit
Tools cost context in two ways, plus one side effect:
- Definitions. Every tool's name, description and parameter schema sits
in context whether or not it is used this turn.
- Intermediate results. Every call puts its full output back into the
conversation, and every round trip is another full model request.
- Side effect: wrong picks. Similar names and vague descriptions make the
model choose the wrong tool or pass wrong arguments.
Tool search mainly saves the first cost. Programmatic tool calling (the model
writes a small program that calls tools) mainly saves the second.
Companion to MCode Engineering Notes 02, "Too many tools will eat your
context". Plain-text checklist: checklists/tool-budget.md.
1. Measure first
Work from the target project's code and real requests, not from guesses:
- List every tool exposed to the model, grouped by source (built-in, each MCP
server, plugins).
- Estimate the token size of each tool definition with a tokenizer and sum it.
Say which tokenizer you used; it is a relative measure.
- For one or two representative tasks, record how many tool calls were made,
how many model requests that caused, and how large the tool results were.
- Note which tools are used almost every turn and which are rarely used.
Report the numbers you measured. If you cannot measure something, say so
instead of estimating.
2. Step 0 - subtract
- Merge by user intent. Do not map APIs one to one. "List issues",
"filter issues by label" and "filter by author" can be one tool with
parameters.
- Make names and descriptions distinguishable. Two tools whose
descriptions read alike will be confused. State the return format.
- Group. Put tools into namespaces by domain, keeping each group small.
- Decide what stays resident. Read, edit, search and run-command style
tools used nearly every turn should always be loaded.
3. Step 1 - deferred loading and tool search
Consider it when there are many tools and most are rarely used.
- Keep core high-frequency tools resident; make low-frequency tools
searchable.
- If a search miss means a tool "does not exist", check descriptions again:
vague descriptions break recall.
- Test tool search recall on your own tool set. Results on clean
benchmarks do not transfer to messy real catalogs.
- Tools whose catalog must be fetched over the network every turn can slow
the first token. Consider keeping them resident instead.
- When debugging, check what the model had actually seen at that moment, not
just what the request listed.
4. Step 2 - programmatic tool calling
Consider it when there is a lot of data to filter or aggregate, three or more
dependent calls, or batch operations that could run in parallel.
- Actually move tools into the program. Offering a code tool next to all the
normal tools tends to go unused.
- Keep at top level: tools that interact with the user, change session state,
need their result read before deciding the next step (tool search, loading
a skill), browser and publishing tools, and spawning subagents.
- Calls from inside the program must go through the same validation, hooks,
permission prompts and UI as direct calls.
- Tool failures should return an error result, not throw, so one failure does
not discard the whole program.
- Prefer structured tool output so the program can pick the fields it needs.
A program that returns raw output saves nothing.
- Search the codebase for logic keyed on tool names (reminders, counters,
analytics). It silently stops firing once a tool moves into the program.
5. Decide
| Bottleneck |
First move |
| Too many or too similar tools |
Subtract, merge, rename, group |
| Definitions take too much context |
Tool search |
| Results take too much context, too many round trips |
Programmatic tool calling |
| Arguments often wrong |
Better descriptions and examples |
The steps stack. A clean catalog makes tool search work; working tool search
makes programmatic calling work. Run an end-to-end eval before and after each
step and look at score, request count and tokens together
(see harness-experiment).
Output
A short report: measured numbers, the main bottleneck, the recommended next
step with concrete tool names, and what to measure afterwards.
1---2name: tool-budget-audit3description: Audit how much context an agent's tools consume, through tool definitions and tool results, and decide whether to trim the catalog, add tool search, or add programmatic tool calling. Use when an agent has many tools or MCP servers, picks the wrong tools, or burns context on tool output.4metadata:5 source: https://skillmd.com/skills/minimax-ai/tool-budget-audit6---78# Tool Budget Audit910Tools cost context in two ways, plus one side effect:1112- **Definitions.** Every tool's name, description and parameter schema sits13 in context whether or not it is used this turn.14- **Intermediate results.** Every call puts its full output back into the15 conversation, and every round trip is another full model request.16- **Side effect: wrong picks.** Similar names and vague descriptions make the17 model choose the wrong tool or pass wrong arguments.1819Tool search mainly saves the first cost. Programmatic tool calling (the model20writes a small program that calls tools) mainly saves the second.2122Companion to MCode Engineering Notes 02, "Too many tools will eat your23context". Plain-text checklist: `checklists/tool-budget.md`.2425## 1. Measure first2627Work from the target project's code and real requests, not from guesses:28291. List every tool exposed to the model, grouped by source (built-in, each MCP30 server, plugins).312. Estimate the token size of each tool definition with a tokenizer and sum it.32 Say which tokenizer you used; it is a relative measure.333. For one or two representative tasks, record how many tool calls were made,34 how many model requests that caused, and how large the tool results were.354. Note which tools are used almost every turn and which are rarely used.3637Report the numbers you measured. If you cannot measure something, say so38instead of estimating.3940## 2. Step 0 - subtract4142- **Merge by user intent.** Do not map APIs one to one. "List issues",43 "filter issues by label" and "filter by author" can be one tool with44 parameters.45- **Make names and descriptions distinguishable.** Two tools whose46 descriptions read alike will be confused. State the return format.47- **Group.** Put tools into namespaces by domain, keeping each group small.48- **Decide what stays resident.** Read, edit, search and run-command style49 tools used nearly every turn should always be loaded.5051## 3. Step 1 - deferred loading and tool search5253Consider it when there are many tools and most are rarely used.5455- Keep core high-frequency tools resident; make low-frequency tools56 searchable.57- If a search miss means a tool "does not exist", check descriptions again:58 vague descriptions break recall.59- Test tool search recall on **your own** tool set. Results on clean60 benchmarks do not transfer to messy real catalogs.61- Tools whose catalog must be fetched over the network every turn can slow62 the first token. Consider keeping them resident instead.63- When debugging, check what the model had actually seen at that moment, not64 just what the request listed.6566## 4. Step 2 - programmatic tool calling6768Consider it when there is a lot of data to filter or aggregate, three or more69dependent calls, or batch operations that could run in parallel.7071- Actually move tools into the program. Offering a code tool next to all the72 normal tools tends to go unused.73- Keep at top level: tools that interact with the user, change session state,74 need their result read before deciding the next step (tool search, loading75 a skill), browser and publishing tools, and spawning subagents.76- Calls from inside the program must go through the same validation, hooks,77 permission prompts and UI as direct calls.78- Tool failures should return an error result, not throw, so one failure does79 not discard the whole program.80- Prefer structured tool output so the program can pick the fields it needs.81 A program that returns raw output saves nothing.82- Search the codebase for logic keyed on tool names (reminders, counters,83 analytics). It silently stops firing once a tool moves into the program.8485## 5. Decide8687| Bottleneck | First move |88| --- | --- |89| Too many or too similar tools | Subtract, merge, rename, group |90| Definitions take too much context | Tool search |91| Results take too much context, too many round trips | Programmatic tool calling |92| Arguments often wrong | Better descriptions and examples |9394The steps stack. A clean catalog makes tool search work; working tool search95makes programmatic calling work. Run an end-to-end eval before and after each96step and look at score, request count and tokens together97(see `harness-experiment`).9899## Output100101A short report: measured numbers, the main bottleneck, the recommended next102step with concrete tool names, and what to measure afterwards.