Context Sweet Spot

Find the Context Size Where Your Sessions Stop Paying

This skill answers one question with the user's own data: at what context size do their coding-agent sessions become a bad deal? Long context is not free even when it fits the window: every cache rebuild re-bills the whole context at write rates, compactions burn turns, and models degrade before their window ends. The sweet spot is where those costs start outrunning the value of the carried context. It is read-only on the platform. Locally it writes a trace export while it works and deletes it again, and leaves one report file behind.

Step 1: Set up the LangWatch CLI

Use the langwatch CLI for everything: documentation (langwatch docs ..., langwatch scenario-docs ...) and platform operations (prompts, scenarios, evaluators, datasets, monitors, traces, analytics). Install it once with npm install -g langwatch, then run the langwatch binary directly; an unpinned npx langwatch re-resolves the package from the registry on every run.

Coding-agent sessions live in the user's personal LangWatch workspace by default. langwatch login --device signs this machine in; add --project <slug> on the read commands when the sessions live in a team project instead.

Use langwatch docs <path> to read documentation as Markdown. Some useful entry points:

langwatch docs                                    # Docs index
langwatch docs integration/python/guide           # Python integration
langwatch docs integration/typescript/guide       # TypeScript integration
langwatch docs prompt-management/cli              # Prompts CLI
langwatch scenario-docs                           # Scenario docs index

Discover commands with langwatch --help and langwatch <subcommand> --help. List and get commands accept --format json for machine-readable output. Every list command takes --limit <n> to cap the rows and --jq <expr> to read part of the answer. A paginated list answers with an envelope, so count its rows through the row array (--jq '.traces | length'), and read how many there are in all at .pagination.total. Bare --jq length counts the fields of the envelope, not the rows. Read the docs first instead of guessing SDK APIs or CLI flags.

If no shell is available, fetch the same Markdown over plain HTTP. Append .md to any docs path (e.g. https://langwatch.ai/docs/integration/python/guide.md). Index: https://langwatch.ai/docs/llms.txt. Scenario index: https://langwatch.ai/scenario/llms.txt

If anything fails or confuses you while following this skill (broken commands, docs that do not match reality, errors you had to work around), ask the user for permission and run npx --yes langwatch report --user-approved with a --title and --summary (or --session <transcript.jsonl>) to send it to the LangWatch team, and it directly shapes what gets fixed. No login or API key needed. Nothing is sent without --user-approved, and --dry-run prints the exact payload without sending anything. The title, summary and transcript are scrubbed locally first, by pattern: secrets and API keys, plus email addresses, phone numbers, card numbers and public IPv4 addresses. Anything no pattern matches is sent as written, including a contact address passed with --email. With --session, always run --dry-run first and let the user read the payload, because a transcript carries content they never reviewed. npx --yes langwatch report --help explains the options.

Step 2: Collect the Sessions

Pick one window and use it everywhere, because trace export defaults to the last 7 days. Compute a start and an end date once (30 days back to now is a good default) and pass both:

langwatch trace export --origin coding_agent --format jsonl --limit 20000 \
  --start-date <start> --end-date <end> -o coding-traces.jsonl

--limit caps the whole export, not one page, so a window with more matches than the limit gives a partial file and the sessions it drops are the ones missing from the buckets. The command reports both counts when it truncates, for example Exported 20000 traces (48213 total). Raise --limit until the two agree, or say in the report that the buckets come from a sample of N of M traces.

Report the window you used in the report, and delete coding-traces.jsonl once the analysis is done.

Each trace carries metadata.thread_id (the session id) and metadata."langwatch.source" (which agent). Collect the distinct session ids, then for each session with enough turns to mean anything (5 or more model calls):

langwatch session events <sessionId> --format json

The events are the raw material: every model call with its input, output, cache-read and cache-creation tokens, its cost, its model, plus explicit compaction and rate_limit events.

Step 3: Compute the Economics

Write a small local script (python3 or jq) over the events. Per session, compute:

1. Peak context: the largest (input + cache-read) of any model call, and its share of the model's context window.
2. Cache rebuilds: model calls whose cache-creation tokens are the bulk of their input, counted only once the session already holds cached context. The first cache-creating call of a session builds the cache rather than rebuilding it, and it pays for the context once, so it is a setup cost and not a rebuild. Count a write from the first call that follows a call with cache-read tokens, or that follows an explicit cache miss. Each rebuild re-paid for context that was already paid for, at the provider's cache-write rate. That rate is specific to the provider and the model: some price a write above fresh input, some price it the same, and some charge for cache storage by time instead. Take it from the price card of the model in question, never from a rule of thumb.
3. Compaction count and where they landed: a compaction late in a session marks the point where the carried context stopped fitting.
4. Cost per model call over session lifetime: split each session into thirds by call order and compare the average cost per call between the first and last third.
5. Waiting time around rebuilds: rebuilt context is also re-uploaded and re-processed, so rebuild-heavy sessions are slower per turn.

Then aggregate across sessions: bucket by peak-context share (for example under 25%, 25 to 50%, 50 to 75%, over 75% of the window) and compare cost per call, rebuild rate and compaction rate between buckets. The sweet spot is the highest bucket where those three stay flat.

Step 4: Report the Finding

Write a single self-contained context-sweet-spot-report.html in the project root (inline CSS, no external assets) with:

• The number: the context share where this user's sessions start degrading, stated in the first line ("your sessions stay economical up to about 55% of the window; past that, cost per turn doubles")
• The bucket comparison table with cost per call, rebuild rate and compaction rate per bucket
• The three most expensive sessions dissected: where the context grew, where it rebuilt, what one rebuild cost
• Concrete habits, each tied to the evidence: when to start a fresh session instead of pushing through, what to offload to sub-agents (the sub-agent keeps its context out of the main session), whether the user's compactions happen late, after the context already crossed the sweet spot
• Links to example sessions in LangWatch for every claim

Also state the top finding directly in the conversation, leading with the number. The LangWatch session detail shows the same cache-health stats per session (/me/sessions), so name it as the place to watch the habit change.

Common Mistakes

• Do NOT judge context by peak share alone; a fat context that never rebuilds is cheap, and a modest one that rebuilds every turn is expensive. The rebuild rate carries the finding.
• Do NOT compare sessions across different models as one population; window sizes and cache pricing differ. Bucket per model, then compare.
• Do NOT count cache-read tokens as cost the way input tokens are; they bill at a fraction. The split is in the event rows, use it.
• Do NOT count the first cache-creating call of a session as a rebuild. It builds the cache, and counting it makes every short session look rebuild-heavy and drags the whole bucket with it.
• Do NOT report a threshold without the sessions behind it; every bucket statistic needs 2 or 3 example session ids.
• Do NOT include sessions with fewer than 5 model calls; they carry no lifetime signal and flatten the buckets.
• If the CLI returns an error, report the user-facing consequence, not the raw error text.