# Trace As State

> Two-pass long-context method: run once to produce a task-state card, then place the card BEFORE the material and re-read to answer. For long-document analysis, multi-hop retrieval, large-codebase comprehension, and any 'you only know what to look for after reading' task. Triggers: long context, multi-hop reasoning, context ordering, state prefixing, trace as state.

- Skill: `wuweifeng710/trace-as-state` (Agent Skill, multi-file: 4 files)
- Install (CLI): `npx skillmds@latest add wuweifeng710/trace-as-state`
- Raw SKILL.md: https://api.skillmd.com/api/skills/wuweifeng710/trace-as-state/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- License: MIT
- Author: WUWeifeng710 (https://skillmd.com/u/wuweifeng710)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/wuweifeng710/trace-as-state

---


# Trace as State — Put the State Before the Material

## 0. In one sentence

Spend one pass over the material, write the resulting **task state** onto a card, **put that card before the material**, and re-read with it in hand before answering.

Order is the only variable. The same trace, placed before vs. after, differs by an order of magnitude.

> **Read §7 before you start.** The essence of this method is **accuracy, not cost**: it trades roughly 2× input tokens for a jump in accuracy, leaves total tokens on par with the control, and adds latency. Cost reduction comes only from the discipline in §7.3 — the method itself does not save tokens. Use it in the wrong setting and you lose.

---

## 1. When to enable

Hit any item on the left → enable. Hit any item on the right → **do not**.

| Enable signals | Do not enable |
|---|---|
| Material exceeds ~1/3 of the context (long docs, large codebases, multiple reports) | Single-hop fact lookup (a value, a definition) |
| The task needs "you only know what to look for after reading" (multi-hop retrieval, "find the clause that contradicts X", "who is most suspicious") | Short input (a few K tokens) |
| State must be maintained across the material (excluded hypotheses, resolved entities, open sub-questions) | Pure generation (writing / translation) with no state to maintain |
| A single long-context call (no history, no cache to sacrifice) | Results insensitive to information order |
| Sub-agent delegation / multi-stage workflow | Multi-turn long session that is **prefix-cache sensitive** (§7.2) — use intra-turn prefixing instead |
| A **hard task a single pass will clearly fail** (long-graph / long edge-list state maintenance, cross-file tracking) | **A first pass that already meets the bar** — gating, §7.3.0 |

**Cost starts at roughly doubled input tokens. With no state to maintain, or when a single pass already suffices, using it is a pure loss.**

---

## 2. Four steps

### Step 1 — Get the trace (reuse first, don't re-run)

**Look for traces that already exist**: searches, greps, full reads, and last turn's conclusions that happened before this turn — those *are* the trace. Excerpt them straight onto the card. **Do not re-run work already done just to "produce a probing pass"** — that is pure waste.

Only when there is genuinely no reasoning trace yet, do one probing pass:

- Process the material normally, **but don't rush to answer** — the goal is a trace, not a finished product
- It may be incomplete and may be wrong — the trace is only a **flawed textual proxy** for task state
- Write down just three things: what I'm looking for / what I ruled out / what I found
- Record anchors along the way (file:line, section number, URL)

### Step 2 — Serialize (write the state card)

Use `templates/state-card.md`. Four hard requirements:

1. **Section 0 must be the task condition** (goal / acceptance criteria / constraints / sub-goal of this pass)
2. Every fact carries an anchor; anything without one is marked "unlocated" and never mixed into established facts
3. It must contain an "Excluded hypotheses" section — this is the part of the state that the *append* order **cannot hold**, and it is the highest-value part
4. The card ends with a disclaimer ("this card may be incomplete or wrong; verify against the source material")

**Keep the card short.** It is compressed state, not another copy of the material. Past ~800 tokens you are copying the source — which is worse than not compressing. The paper truncated traces to 50,000 characters; we deliberately push to ≤800 tokens to contain cost.

### Step 3 — Placement (read §7.2 first to choose)

**Single call (no history) → global prefix:**

```
[state card S] → [source material x] → [current question q]
```

**Multi-turn session (history, cache) → intra-turn prefix** (default):

```
[history, unchanged] → [state card S] → [this turn's new material x] → [this turn's task q]
```

Do not use the paper's control (trace append), and do not reorder the whole history every turn:

```
[source material x] → [state card S] → [current question q]     ← append, the control, worst
[state card S] → [entire history] → [this turn's task]          ← global reorder, cache dies, see §7.2
```

### Step 4 — Verify

The state card is a proxy, not the truth. **Check it line by line against the source material**; where they conflict, the source material wins, and the correction goes into the next card. Skipping this freezes pass-one errors into the answer.

---

## 3. Why this ordering — not folklore

A causal-attention processor recurses as `s_i = U(s_{i-1}, c_i)`, and the condition z (= task state) sets the initial state:

| Ordering | Memory cost | Intuition |
|---|---|---|
| `[z, C]` condition first | Only the **current state** must be stored, ⌈b⌉ bits | Reading with a purpose, pruning as you go |
| `[C, z]` condition last | Must retain a full response profile for **every possible z**, worst case ⌈b·2^b⌉ bits | Reading a 500k-word novel before being told "find the villain" — the whole read was wasted |

> **Ordering principle: the task condition should be available before the information it is meant to guide.**

Note this is **not** the "put important info first" rule of thumb (lost-in-the-middle); it is a structural consequence of causal state updates. The same content placed before beats placed after — strongly enough to beat "pick the best answer after the fact" (Oracle@5).

---

## 4. Anti-patterns (guardrails the paper's ablations hand you directly)

| Anti-pattern | Result | Do instead |
|---|---|---|
| Prefix only the **question** (Question First) | Helps only some tasks (Parents EM 53.0 vs. 81.8 prefixed) | Prefix the **trace/state**, not just the question |
| Only a plain **re-read** (Re2) | Helps (50.0) but far short (81.8) | Re-read **with the state card** |
| Put only the **answer** (Answer Feedback) | Worse than the trace (45.4 vs. 81.8) | The process of the state is worth more than the conclusion |
| Put a **different question's** trace (Random Trace) | **14.2 — worse than no trace (first pass 29.2)** | The trace must belong to the **same task**; cross-task reuse is contamination |
| Trace only, no material (Trace Only) | 43.8 — far below prefixing (81.8) | The source material is **indispensable** |
| Ship the state card as the answer | Errors are inherited | Step 4 verification is not optional |
| Re-do existing work "to run a probing pass" | Burns a full-extra pass of tokens | Step 1 reuse-first |
| Insert the card ahead of the whole history every turn | Prefix cache dies every turn; cost far exceeds the gain | Use intra-turn prefixing (§7.2) |

---

## 5. Agent scenario mapping

The paper is an academic setting (single long-context QA). On an agent it has five landing spots:

| Scenario | How to order |
|---|---|
| **A. Long document / large codebase analysis** | Done within a single call → global prefix: `[state card] → [full text / repo] → [analysis goal]` |
| **B. Multi-turn agent loop** | **Intra-turn prefix**: `[history] → [state card] → [this turn's new material] → [this turn's task]`. Leave history untouched so the cache survives |
| **C. Sub-agent delegation** | Prompt skeleton: `[task condition + goal + constraints + acceptance criteria] → [reference material] → [specific question]`. Most delegation failures come from writing acceptance criteria after the material |
| **D. Multi-stage workflow** (research / review / experiment design) | The previous stage's output is **not** saved for a final summary — it is **prefixed** ahead of the next stage's material. If stages share a long prefix, insert the output only within the stage boundary |
| **E. System prompt / long session** | Keep stable constraints at the very front; don't move them to the end just because "they were emphasized recently" — the tail slot belongs to the current question. This one **adds no cost** (the system prompt is already first and stable) |

---

## 6. The trace-count knob (not cheap)

Probing passes n_tr from 1 to 5 give **generally rising** performance (the paper's word — not strictly monotonic), and the prefix advantage holds throughout (the paper's second pass merges all 5 traces, truncated to 50k characters).

But note: **n_tr = 5 means paying for 5 traces of input.** Raise it only when the task value is high enough and multi-path exploration is genuinely needed. Default n_tr = 1.

---

## 7. The cost ledger (read before you start)

### 7.1 An honest cost model (from the paper's Appendix F, correcting an earlier misread)

| Item | Change | Note |
|---|---|---|
| **Prefix-cache invalidation** | **May force a full recompute every turn** | ⚠️ See §7.2. The largest hidden cost |
| Input tokens | ≈ doubled and up | Two passes + the card; more if n_tr is high |
| Output / reasoning tokens | **Varies by task — not universally down** | Down on GraphWalks (DeepSeek first pass 62.5M → tas 28.0M, append 35.2M); **but up on MRCRv2 / NUB-1M** (DeepSeek MRCRv2 256K: append 0.69M → tas 1.36M) |
| Total tokens | **On par with trace append**, sometimes higher | Appendix F, same order of magnitude; GLM-5.2 GraphWalks total output 35.95M vs. 28.59M |
| Latency | Increased | One extra generation + a longer input |

**Net effect: trace as state is an "accuracy" method, not a "cost" method.** Input roughly doubles, total tokens are on par with the control, and latency goes up. The paper's own Limitations state that it "increases inference latency and token cost."

**Where the gain is largest (consistent with the paper)**: hard tasks where the first pass is weak and the state only surfaces late — multi-hop retrieval, "find the clause that contradicts X", long-graph / long edge-list state maintenance, cross-file tracking in a large codebase. There the Parents-EM-style gain can exceed **+50 points** (DeepSeek GraphWalks Parents EM 29.2 → 81.8). **Spend the accuracy where it pays.**

### 7.2 The key conflict: prefixing vs. cache

The paper's global prefix puts the state card at the **very front of the whole context**, which **changes the prefix**. In a multi-turn conversation the history prefix's KV cache is the main source of cost savings — a cache hit is discounted, a miss is full price.

**Reordering history every turn = the cache dies every turn.** Over a long session this item alone can exceed everything the "output tokens go down" story buys. The paper does note that state prefixing reduces KV-cache reuse in multi-turn settings, but as an aside in the academic setting; on a real agent it is the **primary tension**.

The three placements and their trade-offs:

| Placement | Positional advantage | Cache | Use for |
|---|---|---|---|
| **Global prefix** `[S, x, q]` | Strongest (the paper's original setting) | Lossless when there is no history to sacrifice | Single call, sub-agent delegation, one-shot long-document analysis |
| **Intra-turn prefix** `[history] → [S, x, q]` | Locally preserved (the card still precedes this turn's material) | **Preserved** (the history prefix is unchanged) | **Default for multi-turn agent loops** |
| **Append at the end** `[history] → [x, S, q]` | Weakest (= the paper's control) | Best | Tolerable when the card is tiny (<200 tokens) and the task simple |

**Honest note**: intra-turn prefixing is **not** the paper's global prefix. The paper's mechanism is about state initialization across the entire forward pass; intra-turn prefixing only reproduces the same ordering within **this turn's local causal chain** — directionally consistent, but the **gain may be weaker than global prefixing** and has **no experimental support**. It is an engineering compromise for cache cost, not a paper result.

### 7.3 Four "don't waste it" hard rules (the only source of cost reduction)

> The method does not save cost; cost reduction is discipline: run it only when you can collect accuracy and afford the input, and keep the input minimal.

0. **Gating (most important)**: first judge whether a single pass suffices. If the task is "you only know what to look for after reading" and the first pass already shows gaps / contradictions / open hypotheses → go. If the first pass already meets the bar → **don't** — there is no accuracy to collect, only 2× input to spend.
1. **Reuse first**: excerpt existing traces, don't re-run a probing pass (§2 Step 1)
2. **Keep the card short**: target 200–800 tokens. The paper truncates traces at 50k characters; we go far tighter to contain cost. Past 800, check whether you are copying the source
3. **Intra-turn prefix**: in multi-turn settings, don't touch history (§7.2)
4. **n_tr = 1 by default**: each extra trace is one more input pass; raise it only when the task value is high and the first pass is clearly weak

### 7.4 Where to spend the accuracy (targeting)

Reserve state prefixing for **hard tasks where a single pass clearly fails and the state only emerges late**:
- Multi-hop retrieval / "find the clause that contradicts X" / "who is most suspicious"
- Long-graph / long edge-list state maintenance (the GraphWalks class)
- Cross-file tracking in a large codebase (entities / call chains only assemble after several files)
- Sub-agent delegation where acceptance criteria are written ahead of the material

Soft tasks, tasks a single pass already handles, and short inputs — **let gating reject them.**

---

## 8. Pre-delivery checklist

- [ ] Is Section 0 of the state card the task condition, not a list of facts?
- [ ] Was the trace excerpted from existing reasoning, not re-run for its own sake?
- [ ] Is the state card within 800 tokens?
- [ ] In multi-turn settings, did you use intra-turn prefixing with the history prefix untouched?
- [ ] Does every fact carry an anchor?
- [ ] Is the "Excluded hypotheses" section non-empty?
- [ ] Does the card carry a disclaimer, and has Step 4 run?
- [ ] Is the source material still in context (not degraded to trace-only)?
- [ ] **Did gating pass?** Is this a task the first pass will fail, and were single-pass-sufficient tasks rejected?
- [ ] Does this task really need two passes? (Otherwise the cost is wasted)

---

## Basis

Zou X, Tang J. Trace as State: Reasoning Traces as Conditional States for Long-Context Transformers. arXiv:2609.02702 [cs.CL], 2026-09-02.

Trace as state outperforms trace append in **26 of 27** reported combinations of model, task, and metric. GraphWalks Parents EM: DeepSeek V4 Pro Preview 29.2% → 43.0% (append) → **81.8% (prefix)**; GLM-5.2 66.4% → 83.2% → **100.0%**.

**Cost (Appendix F, the key correction)**: this is an **accuracy, not cost** method — input ≈ doubled, total tokens on par with the control, output tokens task-dependent (down on GraphWalks, up on MRCRv2 / NUB-1M), latency increased. The paper's Limitations explicitly acknowledge that it "increases inference latency and token cost."

Full experimental data, exact ablation figures, confidence intervals, and the token table are in `references/evidence.md` (**load only when checking numbers or when challenged**; not needed for daily use).