Dotnet Observability

dotnet-observability

Modern observability for .NET applications using OpenTelemetry, structured logging, health checks, and custom metrics. Covers the three pillars of observability (traces, metrics, logs), integration with Microsoft.Extensions.Diagnostics and System.Diagnostics, and production-ready health check patterns.

Scope

• OpenTelemetry traces, metrics, and logs setup
• Health check endpoints and liveness/readiness patterns
• Custom metrics with System.Diagnostics.Metrics
• Structured logging integration with OTel export

Out of scope

• DI container mechanics and service lifetimes -- see [skill:dotnet-csharp-dependency-injection]
• Async/await patterns -- see [skill:dotnet-csharp-async-patterns]
• Testing observability output -- see [skill:dotnet-integration-testing]
• Middleware pipeline patterns (request logging, exception handling) -- see [skill:dotnet-middleware-patterns]

Cross-references: [skill:dotnet-csharp-dependency-injection] for service registration, [skill:dotnet-csharp-async-patterns] for async patterns in background exporters, [skill:dotnet-resilience] for Polly telemetry integration, [skill:dotnet-middleware-patterns] for request/exception logging middleware.

OpenTelemetry Setup

OpenTelemetry is the standard observability framework in .NET. The .NET SDK includes native support for System.Diagnostics.Activity (traces) and System.Diagnostics.Metrics (metrics), which OpenTelemetry collects and exports.

Package Landscape

Install the core stack:

<PackageReference Include="OpenTelemetry.Extensions.Hosting" Version="1.*" />
<PackageReference Include="OpenTelemetry.Instrumentation.AspNetCore" Version="1.*" />
<PackageReference Include="OpenTelemetry.Instrumentation.Http" Version="1.*" />
<PackageReference Include="OpenTelemetry.Instrumentation.Runtime" Version="1.*" />
<PackageReference Include="OpenTelemetry.Exporter.OpenTelemetryProtocol" Version="1.*" />

Aspire Service Defaults Integration

If using .NET Aspire, the ServiceDefaults project configures OpenTelemetry automatically. This is the recommended approach for Aspire apps -- do not duplicate this configuration manually:

// ServiceDefaults/Extensions.cs (generated by Aspire)
public static IHostApplicationBuilder AddServiceDefaults(
    this IHostApplicationBuilder builder)
{
    builder.ConfigureOpenTelemetry();
    builder.AddDefaultHealthChecks();
    // ... other defaults
    return builder;
}

For non-Aspire apps, configure OpenTelemetry explicitly as shown below.

Full Configuration (Non-Aspire)

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddOpenTelemetry()
    .ConfigureResource(resource => resource
        .AddService(
            serviceName: builder.Environment.ApplicationName,
            serviceVersion: typeof(Program).Assembly
                .GetCustomAttribute<AssemblyInformationalVersionAttribute>()
                ?.InformationalVersion ?? "unknown"))
    .WithTracing(tracing => tracing
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddSource("MyApp.*")          // Custom ActivitySources
        .AddOtlpExporter())
    .WithMetrics(metrics => metrics
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddRuntimeInstrumentation()
        .AddMeter("MyApp.*")           // Custom Meters
        .AddOtlpExporter());

OTLP Configuration via Environment Variables

The OTLP exporter reads standard environment variables -- no code changes needed between environments:

# Collector endpoint (gRPC default)
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317

# Or HTTP/protobuf
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318

# Resource attributes
OTEL_RESOURCE_ATTRIBUTES=deployment.environment=production,service.namespace=myapp

# Service name (overrides code-based configuration)
OTEL_SERVICE_NAME=order-api

For detailed code examples (custom traces, metrics, structured logging, health checks, production configuration), see examples.md in this skill directory.

Key Principles

• Use OpenTelemetry as the standard -- it provides vendor-neutral instrumentation that works with any backend (Prometheus, Grafana, Datadog, Azure Monitor, AWS X-Ray)
• Use IMeterFactory from DI -- do not create Meter instances directly; the factory integrates with the DI lifecycle and OpenTelemetry registration
• Use source-generated LoggerMessage for hot paths -- zero allocation when the log level is disabled
• Separate liveness from readiness -- liveness checks should not include dependency health; readiness checks should
• Configure via environment variables -- OTLP endpoint, service name, and resource attributes should not be hardcoded
• Enrich logs with trace context -- structured logging with TraceId and SpanId enables log-to-trace correlation
• Follow OpenTelemetry semantic conventions for metric and span names

Agent Gotchas

1. Do not create Meter or ActivitySource via new in DI-registered services without using IMeterFactory -- instruments created outside the factory are not collected by the OpenTelemetry SDK. Use IMeterFactory.Create() for Meter instances. ActivitySource is static and registered via .AddSource().
2. Do not add dependency checks to liveness endpoints -- a database outage should not restart the app. Only the readiness endpoint should check dependencies.
3. Do not use ILogger.LogInformation("message: " + value) or string interpolation $"message: {value}" -- use structured logging templates: ILogger.LogInformation("message: {Value}", value). String concatenation and interpolation bypass structured logging and prevent log indexing.
4. Do not configure OTLP endpoints in code for production -- use environment variables (OTEL_EXPORTER_OTLP_ENDPOINT) so the same image works across environments.
5. Do not forget to register custom ActivitySource names with .AddSource("MyApp.*") -- unregistered sources are silently ignored and produce no traces.

References

• OpenTelemetry .NET documentation
• .NET observability with OpenTelemetry
• ASP.NET Core health checks
• System.Diagnostics.Metrics
• High-performance logging in .NET
• Logging in .NET: Log filtering
• Serilog OpenTelemetry sink

Attribution

Adapted from Aaronontheweb/dotnet-skills (MIT license).