Otel Instrumentation

OpenTelemetry Instrumentation with Kopai

Emit wide events, then prove they are good.

A span is a wide event: one flat record carrying the full context of a unit of work. Kopai runs on localhost, so you never ship instrumentation on faith — you drive traffic through the app yourself and assert on what arrived. The loop is green when every assertion in validate-traces passes. Nothing here is finished while an assertion is red.

Branches

Workflow

Steps 4–6 are a loop, not a sequence. Report the instrumentation complete only when step 5 is green against traffic you drove in step 4 of the same run.

What earns a span

Two questions decide it:

1. Interesting? — does this work meaningfully move latency or failure for the request?
2. Aggregable? — grouped by name and attributes, does it produce a useful trend?

Both yes → create a span. Otherwise → put an attribute on the span you already have.

When in doubt, prefer attributes over child spans. An attribute on the parent needs no JOIN, costs nothing extra to query, and is immediately groupable. Full decision table, plus the three ways this goes wrong, in instrument-spans.

The core pattern

One span, rich attributes, explicit error status. Node.js shown; every language follows the same shape — see your language's rule under Language SDKs below:

import { trace, SpanStatusCode } from "@opentelemetry/api";

const tracer = trace.getTracer("checkout");

export async function processPayment(order: Order) {
  return tracer.startActiveSpan("process-payment", async (span) => {
    try {
      span.setAttributes({
        "order.id": order.id, // IDs are attributes, never span names
        "order.total": order.total,
        "payment.provider": order.provider,
      });
      const receipt = await charge(order);
      span.setAttribute("payment.status", receipt.status);
      return receipt;
    } catch (err) {
      span.recordException(err as Error);
      span.setStatus({
        code: SpanStatusCode.ERROR,
        message: (err as Error).message,
      });
      throw err;
    } finally {
      span.end();
    }
  });
}

Full code for every pattern — nested spans, async fan-out, queues, manual context — in custom-instrumentation.

Naming

• Span names describe the operation: GET /api/users, db.query SELECT, process-payment
• Attribute names are dot-namespaced: user.id, order.total, cache.hit
• Follow OTel semantic conventions for anything standard: http.route, db.system, rpc.service
• Namespace your own additions: app., checkout., <company>.

Span names must be low-cardinality. Never interpolate an ID into a span name — that belongs in an attribute.

Picking instrumentation packages

Detect the project's package manager first: the packageManager field in package.json is authoritative when present; otherwise the lockfile decides — pnpm-lock.yaml → pnpm, yarn.lock → yarn, bun.lock → bun, package-lock.json → npm. Run every install with that manager — a second manager writes a second lockfile and diverges silently. The npx @kopai/cli validation commands are manager-neutral; npx ships with Node and never touches the project's dependency tree.

Check deprecation against the registry, not install output — only npm prints full deprecation warnings at install time; pnpm truncates them and Yarn 4 prints none at all. When npm does print them, read them: never filter an install through tail, grep -v, or --quiet.

npm view @opentelemetry/instrumentation-<lib> deprecated   # empty output = not deprecated

npm view is a pure registry read that works inside pnpm/yarn projects too (npm ships with Node); native equivalents are pnpm view <pkg> deprecated and yarn npm info <pkg> --fields deprecated --json (Yarn 4).

A deprecation message usually names its replacement — use it, never the deprecated package; when none is named, follow the package's own migration guidance instead of guessing. When a framework team ships its own plugin (Fastify → @fastify/otel), prefer it over the contrib package: it registers through the framework's plugin system instead of intercepting require/import — interception that native-ESM imports bypass unless an experimental loader hook is added.

Rules

Task-scoped rule files. Load one only when the workflow points at it.

Setup — setup-backend, setup-environment

Language SDKs — lang-nodejs, lang-fastify, lang-nextjs, lang-python, lang-go, lang-java, lang-dotnet, lang-ruby, lang-php, lang-rust, lang-erlang, lang-cpp

What to instrument — instrument-spans (what earns a span), instrument-attributes (which attributes, timing attributes, async summaries), instrument-errors (error, span status, slug), context-propagation (threading context; framework accessor gotchas), layered-telemetry (traces vs metrics vs logs), sampling

Prove it — drive-traffic (generate the traffic yourself), validate-traces, validate-logs, validate-metrics, validate-shutdown

Fix it — troubleshoot-no-data, troubleshoot-missing-spans, troubleshoot-missing-attrs, troubleshoot-wrong-port

References

Deep dives — load only when the task needs them:

• attributes — the canonical attribute catalog, by category
• custom-instrumentation — full code for every pattern, per language
• architectural-patterns — queues, async fan-out, ETL, serverless
• cli-reference — Kopai CLI commands
• nextjs-examples — Next.js instrumentation examples
• otel-docs — OpenTelemetry documentation links

Related skills

• otel-genai-instrumentation — LLM, agent, and tool-call instrumentation
• root-cause-analysis — investigating telemetry once it is flowing. Instrument for the questions that skill asks: who was affected, what changed, where the time went
• create-dashboard — visualising the signals you emit here