Inspect agent model traffic with LLMTrace
Proxy OpenAI-compatible model traffic so operators can inspect prompts, detect risks, and enforce budget or policy controls.
Prerequisites
Rust or Docker runtime, OpenAI-compatible client or SDK
Installation
Use the upstream install or setup path that matches your environment:
- cargo install llmtrace # from crates.io
- docker pull ghcr.io/epappas/llmtrace-proxy:latest # Docker
- cargo install llmtrace
- pip install llmtracing
Requirements and caveats from upstream:
- python
OpenAI Python SDK
OpenAI Node.js SDK
Basic usage or getting-started notes:
Cost runaway — Uncontrolled API spend, inefficient token usage
Performance Monitoring — Latency, token usage, streaming metrics (TTFT), error tracking
1. Install
Extracted from upstream docs: https://raw.githubusercontent.com/epappas/llmtrace/HEAD/README.md