Utility Skill
Overview
A decision framework for agent orchestration based on Liu et al.,
"Utility-Guided Agent Orchestration for Efficient LLM Tool Use"
(arXiv:2603.19896).
Each candidate action is scored by subtracting weighted costs from
expected gain, producing a single utility value that guides action
selection.
The framework prevents over-calling tools and premature stopping by
making both errors costly.
Utility range is [-2.3, 1.0].
When To Use
- Deciding whether to dispatch another agent or tool call
- Gating expensive tool calls (search, code execution, delegation)
- Selecting the right model tier for a sub-task
- Continuation decisions after receiving partial results
- Verification gating before writing or committing output
When NOT to Use
- Single-step operations with one obvious action
- Trivial tasks where cost of scoring exceeds benefit
- Already-committed actions that cannot be undone
Action Space
A = {respond, retrieve, tool_call, verify, delegate, stop}
| Action |
Description |
| respond |
Emit a final answer from current context |
| retrieve |
Fetch additional information (search, read, lookup) |
| tool_call |
Execute a tool (code runner, API, file write) |
| verify |
Check a prior result for correctness or completeness |
| delegate |
Spawn a sub-agent or hand off to a specialist |
| stop |
Terminate the loop and return current state |
Utility Function
U(a | s_t) = Gain(a | s_t)
- λ₁ · StepCost(a | s_t)
- λ₂ · Uncertainty(a | s_t)
- λ₃ · Redundancy(a | s_t)
| Parameter |
Default |
Rationale |
| λ₁ |
1.0 |
Cost baseline; all other weights relative to this |
| λ₂ |
0.5 |
Weak empirical correlation with outcome (r=0.0131) |
| λ₃ |
0.8 |
Redundancy pruning yields ~10% token savings |
Utility range: [-2.3, 1.0].
Positive values indicate the action is worth taking.
Values below the floor (-0.5 default) indicate the action should
be skipped.
Termination Conditions
Stop the loop when any of the following is true:
- (a) Selected action is
stop
- (b) Step budget exhausted (default: 10 steps)
- (c) All non-
stop actions score below the floor (default: -0.5)
High-gain override: If Gain >= 0.7 for any action, condition
(c) may be overridden.
Document the override and the gain value in your reasoning trace.
Quick Start
Minimal 4-step advisory pattern:
- Construct state: gather task context per
modules/state-builder.md
- Score candidates: evaluate each action in
A per
modules/action-selector.md
- Prefer highest utility: select the action with the
maximum
U(a | s_t), subject to termination conditions
- Log score and decision: record the winning action,
its utility value, and step count before executing
Detailed Resources
- State Builder:
modules/state-builder.md, how to
populate s_t from task context
- Gain:
modules/gain.md, estimating expected information
or progress gain
- Step Cost:
modules/step-cost.md, token, latency, and
monetary cost tables
- Uncertainty:
modules/uncertainty.md, confidence
estimation and calibration
- Redundancy:
modules/redundancy.md, detecting duplicate
or low-delta actions
- Action Selector:
modules/action-selector.md, scoring
loop and tie-breaking rules
- Integration:
modules/integration.md, wiring utility
scoring into existing orchestration loops
Exit Criteria
Source: athola/claude-night-market → plugins/leyline/skills/utility/SKILL.md
1---
2name: utility
3description: "Scores agent actions by expected gain, cost, uncertainty, and redundancy. Use when deciding whether to dispatch an agent or invoke a tool."
4category: ai-agents-and-harness
5source_repo: athola/claude-night-market
6source_path: "plugins/leyline/skills/utility/SKILL.md"
7source_url: https://github.com/athola/claude-night-market/blob/HEAD/plugins/leyline/skills/utility/SKILL.md
8metadata:
9 source: https://skillmd.com/skills/athola/utility
10---
11
12# Utility Skill
13
14## Overview
15
16A decision framework for agent orchestration based on Liu et al.,
17"Utility-Guided Agent Orchestration for Efficient LLM Tool Use"
18(arXiv:2603.19896).
19Each candidate action is scored by subtracting weighted costs from
20expected gain, producing a single utility value that guides action
21selection.
22The framework prevents over-calling tools and premature stopping by
23making both errors costly.
24Utility range is [-2.3, 1.0].
25
26## When To Use
27
28- Deciding whether to dispatch another agent or tool call
29- Gating expensive tool calls (search, code execution, delegation)
30- Selecting the right model tier for a sub-task
31- Continuation decisions after receiving partial results
32- Verification gating before writing or committing output
33
34## When NOT to Use
35
36- Single-step operations with one obvious action
37- Trivial tasks where cost of scoring exceeds benefit
38- Already-committed actions that cannot be undone
39
40## Action Space
41
42`A = {respond, retrieve, tool_call, verify, delegate, stop}`
43
44| Action | Description |
45|-----------|------------------------------------------------------|
46| respond | Emit a final answer from current context |
47| retrieve | Fetch additional information (search, read, lookup) |
48| tool_call | Execute a tool (code runner, API, file write) |
49| verify | Check a prior result for correctness or completeness |
50| delegate | Spawn a sub-agent or hand off to a specialist |
51| stop | Terminate the loop and return current state |
52
53## Utility Function
54
55```
56U(a | s_t) = Gain(a | s_t)
57 - λ₁ · StepCost(a | s_t)
58 - λ₂ · Uncertainty(a | s_t)
59 - λ₃ · Redundancy(a | s_t)
60```
61
62| Parameter | Default | Rationale |
63|-----------|---------|---------------------------------------------------|
64| λ₁ | 1.0 | Cost baseline; all other weights relative to this |
65| λ₂ | 0.5 | Weak empirical correlation with outcome (r=0.0131) |
66| λ₃ | 0.8 | Redundancy pruning yields ~10% token savings |
67
68Utility range: **[-2.3, 1.0]**.
69Positive values indicate the action is worth taking.
70Values below the floor (-0.5 default) indicate the action should
71be skipped.
72
73## Termination Conditions
74
75Stop the loop when **any** of the following is true:
76
77- (a) Selected action is `stop`
78- (b) Step budget exhausted (default: 10 steps)
79- (c) All non-`stop` actions score below the floor (default: -0.5)
80
81**High-gain override:** If `Gain >= 0.7` for any action, condition
82(c) may be overridden.
83Document the override and the gain value in your reasoning trace.
84
85## Quick Start
86
87Minimal 4-step advisory pattern:
88
891. **Construct state**: gather task context per
90 `modules/state-builder.md`
912. **Score candidates**: evaluate each action in `A` per
92 `modules/action-selector.md`
933. **Prefer highest utility**: select the action with the
94 maximum `U(a | s_t)`, subject to termination conditions
954. **Log score and decision**: record the winning action,
96 its utility value, and step count before executing
97
98## Detailed Resources
99
100- **State Builder**: `modules/state-builder.md`, how to
101 populate `s_t` from task context
102- **Gain**: `modules/gain.md`, estimating expected information
103 or progress gain
104- **Step Cost**: `modules/step-cost.md`, token, latency, and
105 monetary cost tables
106- **Uncertainty**: `modules/uncertainty.md`, confidence
107 estimation and calibration
108- **Redundancy**: `modules/redundancy.md`, detecting duplicate
109 or low-delta actions
110- **Action Selector**: `modules/action-selector.md`, scoring
111 loop and tie-breaking rules
112- **Integration**: `modules/integration.md`, wiring utility
113 scoring into existing orchestration loops
114
115## Exit Criteria
116
117- [ ] State constructed with task goal and prior steps
118- [ ] All six actions scored before selecting one
119- [ ] Termination condition checked after each step
120- [ ] Score and decision logged for each step taken
121- [ ] High-gain overrides documented with gain value
122
123---
124
125**Source:** [`athola/claude-night-market`](https://github.com/athola/claude-night-market) → `plugins/leyline/skills/utility/SKILL.md`