modbender Agent Observability
What Gets Installed
| File | Purpose | Location |
|---|---|---|
throughput-dashboard.js |
Weekly productivity metrics | scripts/ |
decision-audit.js |
Append-only decision log with reasoning | lib/ |
failure-tracer.js |
Captures traces when quality score < 7 | lib/ |
drift-guard-auto.js |
Weekly INTENT.md compliance scoring | scripts/ |
Installation
Step 1 — Copy files
WORKSPACE="${OPENCLAW_WORKSPACE:-$(pwd)}"
cp references/throughput-dashboard.js "$WORKSPACE/scripts/"
cp references/decision-audit.js "$WORKSPACE/lib/"
cp references/failure-tracer.js "$WORKSPACE/lib/"
cp references/drift-guard-auto.js "$WORKSPACE/scripts/"
Or manually copy each file from the references/ directory in this skill.
Step 2 — Add to heartbeat/cron (weekly)
In your heartbeat or weekly cron script:
node "$WORKSPACE/scripts/throughput-dashboard.js" "$WORKSPACE"
node "$WORKSPACE/scripts/drift-guard-auto.js" "$WORKSPACE"
Step 3 — Wire decision-audit into high-stakes decisions
const { logDecision } = require('./lib/decision-audit');
logDecision({
task_type: 'code_generation',
decision: 'spawn CoderAgent',
reasoning_summary: 'Multi-file edit blocks chat >5s',
session_channel: 'discord' // optional
}, workspaceRoot);
Step 4 — Wire failure-tracer into quality validation (optional)
The failure-tracer fires automatically when you call it after scoring subagent output:
const { captureFailureTrace } = require('./lib/failure-tracer');
// Call after scoring any subagent output
if (qualityScore < 7) {
captureFailureTrace('AgentLabel-task', qualityScore, agentOutput, workspaceRoot);
}
Reading the Data
| Path | Contents |
|---|---|
memory/dashboards/YYYY-MM-DD.md |
Weekly throughput snapshot |
memory/drift-reports/YYYY-MM-DD.md |
Drift compliance report |
memory/decisions-audit.jsonl |
Full decision log (JSONL) |
memory/traces/[label]-[timestamp].json |
Failure traces |
Query examples
# Recent decisions
tail -20 memory/decisions-audit.jsonl | jq .
# All failure traces
ls memory/traces/
# Latest drift report
cat memory/drift-reports/$(ls memory/drift-reports/ | tail -1)
Tool Descriptions
throughput-dashboard.js
Aggregates weekly metrics: tasks routed, subagents spawned, estimated cost, quality ratio, routing distribution. Reads from session-metrics.js (if installed) and drift-guard-auto.js. Degrades gracefully if data sources are missing — every section is independent.
decision-audit.js
Append-only JSONL log at memory/decisions-audit.jsonl. Each entry: { id, ts, task_type, decision, reasoning_summary, outcome, session_channel }. Use updateOutcome(id, 'success', workspaceRoot) to close the loop after a decision resolves.
failure-tracer.js
Fires when quality score < 7. Writes structured JSON to memory/traces/. Each trace includes: tool call sequence hints, output snippet, inferred failure reason. Use to post-mortem why a subagent underperformed.
drift-guard-auto.js
Scores recent agent outputs against behavioral rules (sycophancy, social cushioning, unprompted explanations, hallucination hedges). Reads INTENT.md for custom criteria if installed. Writes weekly report to memory/drift-reports/.
References
references/throughput-dashboard.js— Full script implementationreferences/decision-audit.js— Full lib implementationreferences/failure-tracer.js— Full lib implementationreferences/drift-guard-auto.js— Full script implementation