1---2name: frontmcp-observability3description: Use when adding tracing, structured logging, metrics, or monitoring to a FrontMCP server. Covers zero-config OpenTelemetry distributed tracing across all flows; the this.telemetry API for custom spans, events, and attributes in tools, plugins, agents, and skills; structured JSON logging with trace correlation and configurable sinks (Winston, Pino, stdout); the off-by-default /metrics endpoint (process and framework metrics, Prometheus-compatible); vendor integrations (Coralogix, Datadog, Logz.io, Grafana Cloud, or any OTLP backend); and testing spans, log correlation, and instrumentation. Triggers: observability, telemetry, tracing, logging, monitoring, OpenTelemetry, OTel, spans, metrics, Prometheus, Datadog, Coralogix, Logz.io, Grafana, Winston, Pino.4license: Apache-2.05---67# FrontMCP Observability89Router for adding observability to FrontMCP servers. Covers distributed tracing (OpenTelemetry), structured JSON logging, per-request log collection, the `this.telemetry` developer API, and vendor integrations.1011## When to Use This Skill1213### Must Use1415- Adding tracing, logging, or monitoring to a FrontMCP server16- Connecting Coralogix, Datadog, Logz.io, Grafana, or any OTLP backend17- Using `this.telemetry` to create custom spans in tools, plugins, or agents18- Setting up structured logging with winston, pino, or NDJSON stdout19- Testing that spans and log entries are created correctly2021### Recommended2223- Before going to production (see also `frontmcp-production-readiness`)24- When debugging request latency or error rates25- When building plugins that need trace context propagation2627### Skip When2829- Building a prototype that doesn't need observability yet30- Configuring auth, transport, or throttle (see `frontmcp-config`)31- Setting up the project from scratch (see `frontmcp-setup`)3233> **Decision:** Use this skill when you need to observe, trace, or log your server. Start with `tracing-setup` for auto-instrumentation, add `structured-logging` for production logs, and use `telemetry-api` for custom spans in your code.3435## Prerequisites3637- A working FrontMCP server (see `frontmcp-setup`)38- `npm install @frontmcp/observability`3940## Step 1: Choose What You Need4142| I want to... | Reference |43| -------------------------------------------------- | ------------------------------------- |44| Enable auto-tracing for all flows | `references/tracing-setup.md` |45| Add structured JSON logging with trace correlation | `references/structured-logging.md` |46| Create custom spans in tools/plugins | `references/telemetry-api.md` |47| Connect Coralogix, Datadog, Logz.io, Grafana | `references/vendor-integrations.md` |48| Test that spans and logs are correct | `references/testing-observability.md` |49| Expose Prometheus `/metrics` endpoint | `references/metrics-endpoint.md` |5051## Step 2: Enable Observability5253The simplest way — one config line:5455```typescript56@FrontMcp({57 observability: true,58})59```6061This enables auto-tracing for all SDK flows. Add structured logging:6263```typescript64@FrontMcp({65 observability: {66 tracing: true,67 logging: { sinks: [{ type: 'stdout' }] },68 requestLogs: true,69 },70})71```7273## Step 3: Read the Relevant Reference7475Follow the scenario routing table above to find the right reference for your use case.7677## Scenario Routing Table7879| Scenario | Reference | Description |80| ------------------------------------- | ------------------------------------- | --------------------------------------------------------------------------- |81| Enable OpenTelemetry tracing | `references/tracing-setup.md` | Zero-config auto-instrumentation, setupOTel(), span hierarchy |82| Add JSON logs with trace correlation | `references/structured-logging.md` | Sinks (stdout, console, OTLP, winston, pino), redaction, log format |83| Custom spans in tools/plugins | `references/telemetry-api.md` | `this.telemetry.startSpan()`, `withSpan()`, `addEvent()`, `setAttributes()` |84| Connect to monitoring platforms | `references/vendor-integrations.md` | Coralogix, Datadog, Logz.io, Grafana — OTLP and direct |85| Test spans and log entries | `references/testing-observability.md` | `createTestTracer()`, `assertSpanExists()`, integration test patterns |86| Expose Prometheus `/metrics` endpoint | `references/metrics-endpoint.md` | Off-by-default Prometheus scrape endpoint with process + framework counters |8788## Common Patterns8990| Pattern | Correct | Incorrect | Why |91| -------------------- | ------------------------------------------- | ------------------------------------------------- | ------------------------------------------------------------- |92| Enable observability | `observability: true` in `@FrontMcp` config | Import and install `ObservabilityPlugin` manually | Config-driven is the standard pattern since v1.0 |93| Custom spans | `this.telemetry.withSpan('op', fn)` | `trace.getTracer().startSpan()` directly | `this.telemetry` auto-inherits trace context |94| Log correlation | `this.logger.info('msg', { key: val })` | `console.log('msg')` | SDK logger flows through StructuredLogTransport with trace_id |95| Session ID | Use `mcp.session.id` attribute (hashed) | Log the real session ID | Privacy: the hash is sufficient for correlation |96| Vendor integration | Use `{ type: 'otlp', endpoint }` sink | Build vendor-specific HTTP clients | OTLP is the universal standard |9798## Quick Reference: What Gets Traced99100| Category | Flows | Attributes |101| -------------- | ---------------------------------------------------- | ------------------------------------------------- |102| HTTP requests | traceRequest, auth, route, finalize | `http.request.method`, `url.path` |103| Tool calls | parseInput → findTool → execute → finalize | `mcp.component.type=tool`, `enduser.id` |104| Resource reads | parseInput → findResource → execute | `mcp.component.type=resource`, `mcp.resource.uri` |105| Prompts | parseInput → findPrompt → execute | `mcp.component.type=prompt` |106| Agents | parseInput → findAgent → execute (nested tool calls) | `mcp.component.type=agent` |107| Auth | verify, session verify, OAuth flows | `frontmcp.auth.mode`, `frontmcp.auth.result` |108| Transport | SSE, Streamable HTTP, Stateless HTTP | `frontmcp.transport.type` |109| Skills | search, load, HTTP endpoints | `frontmcp.flow.name` |110111## Verification Checklist112113### Configuration114115- [ ] `@frontmcp/observability` installed116- [ ] `observability` field added to `@FrontMcp` config117- [ ] TracerProvider configured (via `setupOTel()` or external SDK)118- [ ] Logging sinks configured for production (stdout or OTLP)119120### Runtime121122- [ ] Spans appear in trace backend when calling a tool123- [ ] Log entries include `trace_id` and `span_id`124- [ ] `this.telemetry` is available in tool execution contexts125- [ ] Session tracing ID is consistent across all spans in a request126- [ ] Errors are recorded on spans with `ERROR` status127128### Testing129130- [ ] Tests verify span creation with `createTestTracer()`131- [ ] Tests verify log entries via `CallbackSink`132- [ ] No test isolation issues (each test resets exporter)133134## Troubleshooting135136| Problem | Cause | Solution |137| ------------------------------------------------ | ------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |138| `this.telemetry` is undefined in a tool | `observability` not enabled on the parent `@FrontMcp` config | Set `observability: true` (or a config object) in the `@FrontMcp` decorator; see `tracing-setup` |139| Spans appear without `trace_id` in logs | Logger not connected to `StructuredLogTransport` | Use `this.logger`, not `console`; see `structured-logging` |140| OTLP exporter silently drops spans | Endpoint URL points at the UI, not the OTLP collector | Use the OTLP HTTP/gRPC ingest endpoint exposed by your vendor (Datadog, Coralogix, Logz, etc.); see `vendor-integrations` |141| Real session ID appears in span attributes | A custom span attribute writes `session.id` directly | Use the SDK-provided `mcp.session.id` (already hashed); never log the raw session token |142| Tests randomly fail with leftover spans | Exporter retained between tests | Reset the in-memory exporter in `afterEach`; see `testing-observability` |143| OTel auto-instrumentation double-traces requests | Both `setupOTel()` AND a vendor agent attached to the process | Pick one: either FrontMCP-managed OTel OR the vendor agent — not both |144145## Examples146147Each reference has matching examples under [`examples/<reference>/`](./examples/):148149### `tracing-setup`150151| Example | Level | Description |152| ---------------------------------------------------------------------- | ------------ | ------------------------------------------------------------------------------------------------------ |153| [`basic-tracing`](./examples/tracing-setup/basic-tracing.md) | Basic | Enable auto-tracing and see spans printed to your terminal. |154| [`production-tracing`](./examples/tracing-setup/production-tracing.md) | Intermediate | Full production observability — traces to OTLP, structured logs to stdout, per-request log collection. |155156### `structured-logging`157158| Example | Level | Description |159| ----------------------------------------------------------------------------- | ------------ | ------------------------------------------------------------------------------------------------------------------------------ |160| [`stdout-logging`](./examples/structured-logging/stdout-logging.md) | Basic | Enable NDJSON structured logging to stdout with automatic trace correlation and field redaction. |161| [`winston-integration`](./examples/structured-logging/winston-integration.md) | Intermediate | Forward FrontMCP structured log entries to your existing winston logger. Each entry includes trace_id and span_id as metadata. |162163### `telemetry-api`164165| Example | Level | Description |166| -------------------------------------------------------------------------- | ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------- |167| [`tool-custom-spans`](./examples/telemetry-api/tool-custom-spans.md) | Basic | Create child spans, events, and attributes inside a tool's execute method using this.telemetry. |168| [`plugin-telemetry`](./examples/telemetry-api/plugin-telemetry.md) | Intermediate | Add telemetry events from a custom plugin's hooks. Events appear on the tool execution span, giving you visibility into plugin behavior within the trace. |169| [`agent-nested-tracing`](./examples/telemetry-api/agent-nested-tracing.md) | Advanced | Trace an agent's execution lifecycle including its nested tool calls. Every span shares the same trace ID. |170171### `vendor-integrations`172173| Example | Level | Description |174| ---------------------------------------------------------------------- | ------------ | ------------------------------------------------------------------------------------------------------------------------- |175| [`coralogix-setup`](./examples/vendor-integrations/coralogix-setup.md) | Intermediate | Send both traces and structured logs to Coralogix. Logs include trace_id so Coralogix links them to traces automatically. |176177### `testing-observability`178179| Example | Level | Description |180| ---------------------------------------------------------------------------------- | ------------ | ------------------------------------------------------------------------------------------- |181| [`test-custom-spans`](./examples/testing-observability/test-custom-spans.md) | Basic | Verify that your tool creates the expected child spans with correct attributes. |182| [`test-log-correlation`](./examples/testing-observability/test-log-correlation.md) | Intermediate | Verify that structured log entries include trace context fields for correlation with spans. |183184### `metrics-endpoint`185186| Example | Level | Description |187| ----------------------------------------------------------------------------------- | ----- | -------------------------------------------------------------------- |188| [`enable-metrics-endpoint`](./examples/metrics-endpoint/enable-metrics-endpoint.md) | Basic | Turn on the /metrics endpoint with defaults and scrape it with curl. |189190## Accessing This Skill191192Skills are distributed as plain SKILL.md files plus a sibling `references/`193and `examples/` tree, so consumers can pick whichever access mode fits:194195| Mode | How it works |196| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |197| **Filesystem** | Read `libs/skills/catalog/frontmcp-observability/` directly from a clone of the catalog repo, or from a published `@frontmcp/skills` install. SKILL.md is the entry point. |198| **`frontmcp` CLI** | `frontmcp skills list`, `frontmcp skills read frontmcp-observability`, `frontmcp skills read frontmcp-observability:references/<file>.md`, `frontmcp skills install frontmcp-observability` — no server required. |199| **MCP `skill://`** | When a developer mounts this skill into their own FrontMCP server (`@FrontMcp({ skills: [...] })`), the SDK exposes it via SEP-2640 resources: `skill://frontmcp-observability/SKILL.md`, `skill://frontmcp-observability/references/{file}.md`, etc. The server’s `skill://index.json` returns the SEP-2640 discovery document for everything mounted on it. |200201The catalog itself is **not** an MCP server. The `skill://` URIs only resolve202when a server has been configured to host this skill.203204## Reference205206- [Observability Guide](https://docs.agentfront.dev/frontmcp/guides/observability)207- [Telemetry API Reference](https://docs.agentfront.dev/frontmcp/sdk-reference/telemetry)208- Related skills: `frontmcp-production-readiness`, `frontmcp-config`, `frontmcp-testing`, `frontmcp-development`