MLflow Tracing Instrumentation Guide
Language-Specific Guides
Based on the user's project, load the appropriate guide:
- Python projects: Read
references/python.md
- TypeScript/JavaScript projects: Read
references/typescript.md
If unclear, check for package.json (TypeScript) or requirements.txt/pyproject.toml (Python) in the project.
What to Trace
Trace these operations (high debugging/observability value):
| Operation Type |
Examples |
Why Trace |
| Root operations |
Main entry points, top-level pipelines, workflow steps |
End-to-end latency, input/output logging |
| LLM calls |
Chat completions, embeddings |
Token usage, latency, prompt/response inspection |
| Retrieval |
Vector DB queries, document fetches, search |
Relevance debugging, retrieval quality |
| Tool/function calls |
API calls, database queries, web search |
External dependency monitoring, error tracking |
| Agent decisions |
Routing, planning, tool selection |
Understand agent reasoning and choices |
| External services |
HTTP APIs, file I/O, message queues |
Dependency failures, timeout tracking |
Skip tracing these (too granular, adds noise):
- Simple data transformations (dict/list manipulation)
- String formatting, parsing, validation
- Configuration loading, environment setup
- Logging or metric emission
- Pure utility functions (math, sorting, filtering)
Rule of thumb: Trace operations that are important for debugging and identifying issues in your application.
Feedback Collection
Log user feedback on traces for evaluation, debugging, and fine-tuning. Essential for identifying quality issues in production.
See references/feedback-collection.md for:
- Recording user ratings and comments with
mlflow.log_feedback()
- Capturing trace IDs to return to clients
- LLM-as-judge automated evaluation
Reference Documentation
Production Deployment
See references/production.md for:
- Environment variable configuration
- Async logging for low-latency applications
- Sampling configuration (MLFLOW_TRACE_SAMPLING_RATIO)
- Lightweight SDK (
mlflow-tracing)
- Docker/Kubernetes deployment
Advanced Patterns
See references/advanced-patterns.md for:
- Async function tracing
- Multi-threading with context propagation
- PII redaction with span processors
Distributed Tracing
See references/distributed-tracing.md for:
- Propagating trace context across services
- Client/server header APIs
1---2name: instrumenting-with-mlflow-tracing3description: Instruments Python and TypeScript code with MLflow Tracing for observability. Triggers on questions about adding tracing, instrumenting agents/LLM apps, getting started with MLflow tracing, or tracing specific frameworks (LangGraph, LangChain, OpenAI, DSPy, CrewAI, AutoGen). Examples - "How do I add tracing?", "How to instrument my agent?", "How to trace my LangChain app?", "Getting started with MLflow tracing", "Trace my TypeScript app"4---56# MLflow Tracing Instrumentation Guide78## Language-Specific Guides910Based on the user's project, load the appropriate guide:1112- **Python projects**: Read `references/python.md`13- **TypeScript/JavaScript projects**: Read `references/typescript.md`1415If unclear, check for `package.json` (TypeScript) or `requirements.txt`/`pyproject.toml` (Python) in the project.1617---1819## What to Trace2021**Trace these operations** (high debugging/observability value):2223| Operation Type | Examples | Why Trace |24|---------------|----------|-----------|25| **Root operations** | Main entry points, top-level pipelines, workflow steps | End-to-end latency, input/output logging |26| **LLM calls** | Chat completions, embeddings | Token usage, latency, prompt/response inspection |27| **Retrieval** | Vector DB queries, document fetches, search | Relevance debugging, retrieval quality |28| **Tool/function calls** | API calls, database queries, web search | External dependency monitoring, error tracking |29| **Agent decisions** | Routing, planning, tool selection | Understand agent reasoning and choices |30| **External services** | HTTP APIs, file I/O, message queues | Dependency failures, timeout tracking |3132**Skip tracing these** (too granular, adds noise):3334- Simple data transformations (dict/list manipulation)35- String formatting, parsing, validation36- Configuration loading, environment setup37- Logging or metric emission38- Pure utility functions (math, sorting, filtering)3940**Rule of thumb**: Trace operations that are important for debugging and identifying issues in your application.4142---4344## Feedback Collection4546Log user feedback on traces for evaluation, debugging, and fine-tuning. Essential for identifying quality issues in production.4748See `references/feedback-collection.md` for:49- Recording user ratings and comments with `mlflow.log_feedback()`50- Capturing trace IDs to return to clients51- LLM-as-judge automated evaluation5253---5455## Reference Documentation5657### Production Deployment5859See `references/production.md` for:60- Environment variable configuration61- Async logging for low-latency applications62- Sampling configuration (MLFLOW_TRACE_SAMPLING_RATIO)63- Lightweight SDK (`mlflow-tracing`)64- Docker/Kubernetes deployment6566### Advanced Patterns6768See `references/advanced-patterns.md` for:69- Async function tracing70- Multi-threading with context propagation71- PII redaction with span processors7273### Distributed Tracing7475See `references/distributed-tracing.md` for:76- Propagating trace context across services77- Client/server header APIs