Utility Skill
Overview
A decision framework for agent orchestration based on Liu et al.,
"Utility-Guided Agent Orchestration for Efficient LLM Tool Use"
(arXiv:2603.19896).
Each candidate action is scored by subtracting weighted costs from
expected gain, producing a single utility value that guides action
selection.
The framework prevents over-calling tools and premature stopping by
making both errors costly.
Utility range is [-2.3, 1.0].
When To Use
- Deciding whether to dispatch another agent or tool call
- Gating expensive tool calls (search, code execution, delegation)
- Selecting the right model tier for a sub-task
- Continuation decisions after receiving partial results
- Verification gating before writing or committing output
When NOT to Use
- Single-step operations with one obvious action
- Trivial tasks where cost of scoring exceeds benefit
- Already-committed actions that cannot be undone
Action Space
A = {respond, retrieve, tool_call, verify, delegate, stop}
| Action |
Description |
| respond |
Emit a final answer from current context |
| retrieve |
Fetch additional information (search, read, lookup) |
| tool_call |
Execute a tool (code runner, API, file write) |
| verify |
Check a prior result for correctness or completeness |
| delegate |
Spawn a sub-agent or hand off to a specialist |
| stop |
Terminate the loop and return current state |
Utility Function
U(a | s_t) = Gain(a | s_t)
- λ₁ · StepCost(a | s_t)
- λ₂ · Uncertainty(a | s_t)
- λ₃ · Redundancy(a | s_t)
| Parameter |
Default |
Rationale |
| λ₁ |
1.0 |
Cost baseline; all other weights relative to this |
| λ₂ |
0.5 |
Weak empirical correlation with outcome (r=0.0131) |
| λ₃ |
0.8 |
Redundancy pruning yields ~10% token savings |
Utility range: [-2.3, 1.0].
Positive values indicate the action is worth taking.
Values below the floor (-0.5 default) indicate the action should
be skipped.
Termination Conditions
Stop the loop when any of the following is true:
- (a) Selected action is
stop
- (b) Step budget exhausted (default: 10 steps)
- (c) All non-
stop actions score below the floor (default: -0.5)
High-gain override: If Gain >= 0.7 for any action, condition
(c) may be overridden.
Document the override and the gain value in your reasoning trace.
Quick Start
Minimal 4-step advisory pattern:
- Construct state: gather task context per
modules/state-builder.md
- Score candidates: evaluate each action in
A per
modules/action-selector.md
- Prefer highest utility: select the action with the
maximum
U(a | s_t), subject to termination conditions
- Log score and decision: record the winning action,
its utility value, and step count before executing
Detailed Resources
- State Builder:
modules/state-builder.md, how to
populate s_t from task context
- Gain:
modules/gain.md, estimating expected information
or progress gain
- Step Cost:
modules/step-cost.md, token, latency, and
monetary cost tables
- Uncertainty:
modules/uncertainty.md, confidence
estimation and calibration
- Redundancy:
modules/redundancy.md, detecting duplicate
or low-delta actions
- Action Selector:
modules/action-selector.md, scoring
loop and tie-breaking rules
- Integration:
modules/integration.md, wiring utility
scoring into existing orchestration loops
Exit Criteria
Source: athola/claude-night-market → plugins/leyline/skills/utility/SKILL.md
1---2name: utility3description: Scores agent actions by expected gain, cost, uncertainty, and redundancy. Use when deciding whether to dispatch an agent or invoke a tool.4---5
6# Utility Skill
7
8## Overview
9
10A decision framework for agent orchestration based on Liu et al.,
11"Utility-Guided Agent Orchestration for Efficient LLM Tool Use"
12(arXiv:2603.19896).
13Each candidate action is scored by subtracting weighted costs from
14expected gain, producing a single utility value that guides action
15selection.
16The framework prevents over-calling tools and premature stopping by
17making both errors costly.
18Utility range is [-2.3, 1.0].
19
20## When To Use
21
22- Deciding whether to dispatch another agent or tool call
23- Gating expensive tool calls (search, code execution, delegation)
24- Selecting the right model tier for a sub-task
25- Continuation decisions after receiving partial results
26- Verification gating before writing or committing output
27
28## When NOT to Use
29
30- Single-step operations with one obvious action
31- Trivial tasks where cost of scoring exceeds benefit
32- Already-committed actions that cannot be undone
33
34## Action Space
35
36`A = {respond, retrieve, tool_call, verify, delegate, stop}`
37
38| Action | Description |
39|-----------|------------------------------------------------------|
40| respond | Emit a final answer from current context |
41| retrieve | Fetch additional information (search, read, lookup) |
42| tool_call | Execute a tool (code runner, API, file write) |
43| verify | Check a prior result for correctness or completeness |
44| delegate | Spawn a sub-agent or hand off to a specialist |
45| stop | Terminate the loop and return current state |
46
47## Utility Function
48
49```
50U(a | s_t) = Gain(a | s_t)
51 - λ₁ · StepCost(a | s_t)
52 - λ₂ · Uncertainty(a | s_t)
53 - λ₃ · Redundancy(a | s_t)
54```
55
56| Parameter | Default | Rationale |
57|-----------|---------|---------------------------------------------------|
58| λ₁ | 1.0 | Cost baseline; all other weights relative to this |
59| λ₂ | 0.5 | Weak empirical correlation with outcome (r=0.0131) |
60| λ₃ | 0.8 | Redundancy pruning yields ~10% token savings |
61
62Utility range: **[-2.3, 1.0]**.
63Positive values indicate the action is worth taking.
64Values below the floor (-0.5 default) indicate the action should
65be skipped.
66
67## Termination Conditions
68
69Stop the loop when **any** of the following is true:
70
71- (a) Selected action is `stop`
72- (b) Step budget exhausted (default: 10 steps)
73- (c) All non-`stop` actions score below the floor (default: -0.5)
74
75**High-gain override:** If `Gain >= 0.7` for any action, condition
76(c) may be overridden.
77Document the override and the gain value in your reasoning trace.
78
79## Quick Start
80
81Minimal 4-step advisory pattern:
82
831. **Construct state**: gather task context per
84 `modules/state-builder.md`
852. **Score candidates**: evaluate each action in `A` per
86 `modules/action-selector.md`
873. **Prefer highest utility**: select the action with the
88 maximum `U(a | s_t)`, subject to termination conditions
894. **Log score and decision**: record the winning action,
90 its utility value, and step count before executing
91
92## Detailed Resources
93
94- **State Builder**: `modules/state-builder.md`, how to
95 populate `s_t` from task context
96- **Gain**: `modules/gain.md`, estimating expected information
97 or progress gain
98- **Step Cost**: `modules/step-cost.md`, token, latency, and
99 monetary cost tables
100- **Uncertainty**: `modules/uncertainty.md`, confidence
101 estimation and calibration
102- **Redundancy**: `modules/redundancy.md`, detecting duplicate
103 or low-delta actions
104- **Action Selector**: `modules/action-selector.md`, scoring
105 loop and tie-breaking rules
106- **Integration**: `modules/integration.md`, wiring utility
107 scoring into existing orchestration loops
108
109## Exit Criteria
110
111- [ ] State constructed with task goal and prior steps
112- [ ] All six actions scored before selecting one
113- [ ] Termination condition checked after each step
114- [ ] Score and decision logged for each step taken
115- [ ] High-gain overrides documented with gain value
116
117---
118
119**Source:** [`athola/claude-night-market`](https://github.com/athola/claude-night-market) → `plugins/leyline/skills/utility/SKILL.md`
thedixitjain