Debug Instrumentation

Debug Your LangWatch Instrumentation

This recipe uses the langwatch CLI to inspect your production traces and identify instrumentation issues.

Prerequisites

Use langwatch docs <path> to read documentation as Markdown. Some useful entry points:

langwatch docs                                    # Docs index
langwatch docs integration/python/guide           # Python integration
langwatch docs integration/typescript/guide       # TypeScript integration
langwatch docs prompt-management/cli              # Prompts CLI
langwatch scenario-docs                           # Scenario docs index

Discover commands with langwatch --help and langwatch <subcommand> --help. List and get commands accept --format json for machine-readable output. Every list command takes --limit <n> to cap the rows and --jq <expr> to read part of the answer. A paginated list answers with an envelope, so count its rows through the row array (--jq '.traces | length'), and read how many there are in all at .pagination.total. Bare --jq length counts the fields of the envelope, not the rows. Read the docs first instead of guessing SDK APIs or CLI flags.

If no shell is available, fetch the same Markdown over plain HTTP. Append .md to any docs path (e.g. https://langwatch.ai/docs/integration/python/guide.md). Index: https://langwatch.ai/docs/llms.txt. Scenario index: https://langwatch.ai/scenario/llms.txt

If anything fails or confuses you while following this skill (broken commands, docs that do not match reality, errors you had to work around), ask the user for permission and run npx --yes langwatch report --user-approved with a --title and --summary (or --session <transcript.jsonl>) to send it to the LangWatch team, and it directly shapes what gets fixed. No login or API key needed. Nothing is sent without --user-approved, and --dry-run prints the exact payload without sending anything. The title, summary and transcript are scrubbed locally first, by pattern: secrets and API keys, plus email addresses, phone numbers, card numbers and public IPv4 addresses. Anything no pattern matches is sent as written, including a contact address passed with --email. With --session, always run --dry-run first and let the user read the payload, because a transcript carries content they never reviewed. npx --yes langwatch report --help explains the options.

Step 1: Fetch Recent Traces

langwatch trace search --limit 25 --start-date "$(( ($(date +%s) - 7*24*3600) * 1000 ))" --format json

(Widen or narrow the window as needed. --start-date accepts an ISO string or epoch milliseconds, and defaults to the last 24 hours. The epoch form above is used because date -d '7 days ago' is GNU-only and fails on macOS.)

For each trace, ask:

• How many traces are there?
• Do they have inputs and outputs populated, or are they <empty>?
• Are there labels and metadata (user_id, thread_id)?

langwatch status is a fast sanity check that the CLI is talking to the right project.

Step 2: Inspect Individual Traces

langwatch trace get <traceId>            # Human-readable digest
langwatch trace get <traceId> -f json    # Full span hierarchy as JSON

For traces that look problematic, check for:

• Empty input/output: The most common issue. Check if autotrack_openai_calls(client) (Python) or experimental_telemetry (TypeScript/Vercel AI) is configured.
• Disconnected spans: Spans that don't connect to a parent trace. Usually means @langwatch.trace() decorator is missing on the entry function.
• Missing labels: No way to filter traces by feature/version. Add labels via langwatch.get_current_trace().update(metadata={"labels": ["feature_name"]}).
• Missing user_id/thread_id: Can't correlate traces to users or conversations. Add via trace metadata.
• Slow spans: Unusually long completion times may indicate API timeouts or inefficient prompts.

Step 3: Read the Integration Docs

Use the CLI to read the integration guide for the project's framework. Compare the recommended setup with what's in the code.

langwatch docs                                  # Browse the docs index
langwatch docs integration/python/guide         # Python (or your framework)
langwatch docs integration/typescript/guide     # TypeScript (or your framework)

Step 4: Apply Fixes

For each issue found:

1. Identify the root cause in the code
2. Apply the fix following the framework-specific docs
3. Run the application to generate new traces
4. Re-inspect with langwatch trace search and langwatch trace get to verify the fix

Step 5: Verify Improvement

After fixes, compare before/after:

• Are inputs/outputs now populated?
• Are spans properly nested?
• Are labels and metadata present?

You can also export a sample for diff:

langwatch trace export --format jsonl --limit 50 -o traces.jsonl

Common Issues and Fixes