Otel Tracing Pattern

OpenTelemetry Tracing Pattern

Instrument distributed traces with OpenTelemetry spans to visualize request flow across services

When to Use

• Debugging slow requests across microservices
• Understanding which service in a chain is causing latency
• Correlating logs and errors to specific request traces
• Building visibility into async workflows (queues, background jobs)

Instructions

1. Get a tracer from the registered TracerProvider: trace.getTracer('service-name', '1.0.0').
2. Create spans for significant operations: HTTP handlers, database queries, external API calls, queue processing.
3. Use tracer.startActiveSpan to automatically propagate context to child spans.
4. Set span attributes for searchability: span.setAttribute('user.id', userId).
5. Set span status on error: span.setStatus({ code: SpanStatusCode.ERROR, message }).
6. Always end spans in a finally block — unfinished spans leak memory.
7. Use semantic conventions for attribute names (http.method, db.system, rpc.method).

// services/order-service.ts
import { trace, SpanStatusCode, SpanKind } from '@opentelemetry/api';

const tracer = trace.getTracer('order-service', '1.0.0');

export async function createOrder(userId: string, items: OrderItem[]): Promise<Order> {
  return tracer.startActiveSpan(
    'createOrder',
    {
      kind: SpanKind.INTERNAL,
      attributes: {
        'user.id': userId,
        'order.item_count': items.length,
      },
    },
    async (span) => {
      try {
        // Child span — automatically linked to parent via active context
        const inventory = await checkInventory(items);

        const order = await tracer.startActiveSpan('db.insertOrder', async (dbSpan) => {
          try {
            dbSpan.setAttribute('db.system', 'postgresql');
            dbSpan.setAttribute('db.operation', 'INSERT');
            const result = await db.orders.create({ userId, items, inventory });
            return result;
          } finally {
            dbSpan.end();
          }
        });

        span.setAttribute('order.id', order.id);
        span.setStatus({ code: SpanStatusCode.OK });
        return order;
      } catch (error) {
        span.setStatus({
          code: SpanStatusCode.ERROR,
          message: error instanceof Error ? error.message : 'Unknown error',
        });
        span.recordException(error as Error);
        throw error;
      } finally {
        span.end();
      }
    }
  );
}

Details

Span hierarchy: Traces are trees of spans. The root span represents the entire request. Child spans represent sub-operations. The span context (trace ID + span ID) propagates automatically when using startActiveSpan.

Span kinds:

• SERVER — handling an incoming request
• CLIENT — making an outgoing request
• INTERNAL — internal operation (default)
• PRODUCER — sending a message to a queue
• CONSUMER — receiving a message from a queue

Semantic conventions (use these attribute names for tool compatibility):

span.setAttribute('http.method', 'POST');
span.setAttribute('http.url', '/api/orders');
span.setAttribute('http.status_code', 201);
span.setAttribute('db.system', 'postgresql');
span.setAttribute('db.statement', 'INSERT INTO orders ...');
span.setAttribute('messaging.system', 'rabbitmq');
span.setAttribute('messaging.destination', 'order-events');

Span events: Add timestamped events within a span for notable occurrences:

span.addEvent('inventory.checked', {
  'inventory.available': true,
  'inventory.reserved_count': 5,
});

Common mistakes:

• Forgetting to end spans (memory leak, incomplete traces)
• Creating too many spans (every function call) — trace backends charge per span
• Not setting error status (errors appear as successful spans in dashboards)
• Using non-standard attribute names (breaks dashboard queries)

Source

https://opentelemetry.io/docs/concepts/signals/traces/

Process

1. Read the instructions and examples in this document.
2. Apply the patterns to your implementation, adapting to your specific context.
3. Verify your implementation against the details and edge cases listed above.

Harness Integration

• Type: knowledge — this skill is a reference document, not a procedural workflow.
• No tools or state — consumed as context by other skills and agents.

Success Criteria

• The patterns described in this document are applied correctly in the implementation.
• Edge cases and anti-patterns listed in this document are avoided.