Atlas Optimize

atlas-optimize

The efficiency counterpart to draco-analyze. That skill asks did the research score well; this one asks where did the time and money go, and which function should change.

It reads the trace --trace captures for a run: per-call / per-agent / per-phase timing spans (with a compute-vs-gate-wait split), an automatic bottleneck digest (critical path, phase + per-agent rollups, concurrency peaks, ranked anomalies), and the byte-exact model transcript — every step's system prompt, input thread, thinking, and tool calls. Crucially the digest carries an attribution map (site → src/file.ts:fn), so every hot span points straight at the code that owns it. That map is the whole reason this is a skill and not an SDK feature: a finding maps to a file you can open and read.

Producing a trace

A run only emits a trace when started with --trace. Traces land under eval-runs/traces/<commit>/ (git-ignored), one pair of files per run: <runId>.digest.json (small) and <runId>.trace.json (spans + transcript).

tsx examples/cli.ts "<question>" --trace full --effort balanced

• --trace full = spans + verbatim transcript. --trace spans = the lighter tier (timing + digest, no verbatim I/O) — enough for bottleneck work, cheaper to keep.
• If the user names a question to optimize and no trace exists, offer to run this and proceed only on confirmation — it spends provider + Steel credits.
• If they say "analyze this commit's traces", read what's already on disk; don't re-run.

Reading traces — one keyless CLI

All reads go through tsx examples/trace.ts. The data is large, so the rule is progressive disclosure: start at the digest, descend only as far as the question needs. Never dump a full transcript blind — slice it.

tsx examples/trace.ts commits                 # commit dirs that have traces
tsx examples/trace.ts list [--commit SHA]     # runs for a commit (default HEAD): wall/cost/waitRatio/peak/top anomaly
tsx examples/trace.ts digest <runId>          # the bottleneck digest — START HERE
tsx examples/trace.ts spans <runId> [--kind model|tool|io|agent] [--grep RE]
tsx examples/trace.ts transcript <runId> [--role R] [--seq A-B] [--step N] [--grep RE] [--head N] [--messages]

Output is JSON (transcript renders text); parse it, don't echo it at the user.

Step 0 — Resolve the commit and the runs

If the user gave a runId, use it. Otherwise default to the current commit (git rev-parse --short HEAD) and list its runs; say which you used. If the commit has no traces, say so and offer to produce one. With several runs of the same question, treat them as repeated samples — timing varies run to run.

Step 1 — Read the digest (the headline)

tsx examples/trace.ts digest <runId>

Read off, in order:

• criticalPath + criticalPathMs — the chain of spans that actually fills the wall clock. This is where time went. Each entry carries its site.
• waitVsCompute — { computeMs, waitMs, ratio }. ratio ≫ 1 means most model time was spent queued for the gate, not computing.
• concurrency — peakModelInFlight vs gateLimitModel. Peak pinned at the limit and high wait ⇒ gate-starved.
• anomalies — pre-ranked by severity; each has a site and a detail.
• phaseBreakdown (keyed by site) and byAgent — where wall / compute / cost / tokens concentrate. byAgent splits selfMs (own model time) from subtreeMs (incl. children).
• topByWait / topByLatency / topByCost — the worst individual model calls.
• attribution — site → src/file.ts:fn. This is your jump table to code.

Step 2 — Symptom → cause → source

Always resolve the file from the digest's own attribution map — don't guess the filename from the site name.

Step 3 — Drill only where a hypothesis needs it

tsx examples/trace.ts spans <runId> --kind model --grep verify     # one phase's calls: waitMs, computeMs, cost
tsx examples/trace.ts transcript <runId>                            # summary: steps per role
tsx examples/trace.ts transcript <runId> --role lead               # the lead's reasoning + tool calls
tsx examples/trace.ts transcript <runId> --seq 8-10 --messages     # byte-exact input for a step range (large)

Use the transcript to answer what the aggregates can't: why is this call's prompt so big? what is this agent re-reading every step? did synthesis get handed redundant context? Then open the attributed source and read it before proposing anything.

Step 4 — Synthesize: bottlenecks → fixes

Produce, for the user:

1. Ranked bottlenecks — each tied to a digest signal (quote the number: waitMs, share of criticalPathMs, costUSD), the root cause, and the src/file.ts:fn it lives in (from attribution).
2. A concrete fix per bottleneck — read the file first, then cite file:line and the change. Separate config fixes (a knob in config.ts, low-risk) from structural fixes (a pipeline change — overlap awaits, memoize a call, trim a prompt, rebalance fan-out).
3. Cross-run patterns — if several runs on this commit show the same hot site, that's the highest-leverage fix; a site that's hot in one run but not others is variance, not a bug.

Step 5 — Close the loop (verify a fix)

After a fix is applied (by the user, or in a normal edit turn — not under this skill), confirm it moved the needle:

tsx examples/cli.ts "<same question>" --trace full --effort <same>
tsx examples/trace.ts digest <newRunId>

Diff the new digest against the old: did criticalPathMs / modelWaitMs / costUSD drop, did the anomaly clear, and where did the bottleneck move next (it usually moves — name the new long pole)? If src/ changed, confirm npm run test:run is green. Report before/after numbers, not just "fixed".

Boundaries

• Read-only. This skill diagnoses and proposes; it does not edit src/. Hand off each fix as file:line + the change. If asked to apply it, do so in a normal turn, not under this skill.
• Progressive disclosure. Start at digest; slice spans / transcript. Never dump a whole transcript to reach a conclusion.
• Replayed work isn't a bottleneck. Replayed calls have zeroed timing and status:"replayed"; the digest already excludes them from latency math — don't re-introduce them as if they cost wall-clock.
• Honor variance. Timing shifts with network and provider load. One run is a hypothesis; if numbers swing across runs of the same question, say so rather than over-fitting a story to a single run.
• Ground every claim. Quote the digest field, the span's waitMs/computeMs, the transcript line, or the file:line you cite. If the trace can't decide, say what's missing (e.g. "this run was --trace spans; need full to see the prompt").
• Mind cost. Producing a trace runs real research. Re-run to verify only when it earns the spend; prefer reusing the commit's existing traces.