rudironsoni dotnet-profiling
Diagnostic tool guidance for investigating .NET performance problems. Covers real-time metric monitoring with dotnet-counters, event tracing and flame graph generation with dotnet-trace, and memory dump capture and analysis with dotnet-dump. Focuses on interpreting profiling data (reading flame graphs, analyzing heap dumps, correlating GC metrics) rather than just invoking tools.
Version assumptions: .NET SDK 8.0+ baseline. All three diagnostic tools (dotnet-counters, dotnet-trace, dotnet-dump) ship with the .NET SDK -- no separate installation required.
Scope
- Real-time metric monitoring with dotnet-counters
- Event tracing and flame graph generation with dotnet-trace
- Memory dump capture and analysis with dotnet-dump
- Interpreting profiling data (flame graphs, heap dumps, GC metrics)
Out of scope
- OpenTelemetry metrics and distributed tracing -- see [skill:dotnet-observability]
- Microbenchmarking setup (BenchmarkDotNet) -- see [skill:dotnet-benchmarkdotnet]
- Performance architecture patterns (Span, ArrayPool, sealed) -- see [skill:dotnet-performance-patterns]
- Continuous benchmark regression detection in CI -- see [skill:dotnet-ci-benchmarking]
- Architecture patterns (caching, resilience) -- see [skill:dotnet-architecture-patterns]
Cross-references: [skill:dotnet-observability] for GC/threadpool metrics interpretation and OpenTelemetry correlation, [skill:dotnet-benchmarkdotnet] for structured benchmarking after profiling identifies hot paths, [skill:dotnet-performance-patterns] for optimization patterns to apply based on profiling results.
dotnet-counters -- Real-Time Metric Monitoring
Overview
dotnet-counters provides real-time monitoring of .NET runtime metrics without modifying application code. Use it as a
first-pass triage tool to identify whether a performance problem is CPU-bound, memory-bound, or I/O-bound before
reaching for heavier instrumentation.
Monitoring Running Processes
# List running .NET processes
dotnet-counters ps
# Monitor default runtime counters for a process
dotnet-counters monitor --process-id <PID>
# Monitor with a specific refresh interval (seconds)
dotnet-counters monitor --process-id <PID> --refresh-interval 2
```text
### Key Built-In Counter Providers
| Provider | Counters | What It Tells You |
|----------|----------|-------------------|
| `System.Runtime` | CPU usage, GC heap size, Gen 0/1/2 collections, threadpool queue length, exception count | Overall runtime health |
| `Microsoft.AspNetCore.Hosting` | Request rate, request duration, active requests | HTTP request throughput and latency |
| `Microsoft.AspNetCore.Http.Connections` | Connection duration, current connections | WebSocket/SignalR connection load |
| `System.Net.Http` | Requests started/failed, active requests, connection pool size | Outbound HTTP client behavior |
| `System.Net.Sockets` | Bytes sent/received, datagrams, connections | Network I/O volume |
### Monitoring Specific Providers
```bash
# Monitor runtime and ASP.NET counters together
dotnet-counters monitor --process-id <PID> \
--counters System.Runtime,Microsoft.AspNetCore.Hosting
# Monitor only GC-related counters
dotnet-counters monitor --process-id <PID> \
--counters System.Runtime[gc-heap-size,gen-0-gc-count,gen-1-gc-count,gen-2-gc-count]
```text
### Custom EventCounters
Applications can publish custom counters for domain-specific metrics:
```csharp
using System.Diagnostics.Tracing;
[EventSource(Name = "MyApp.Orders")]
public sealed class OrderMetrics : EventSource
{
public static readonly OrderMetrics Instance = new();
private EventCounter? _orderProcessingTime;
private IncrementingEventCounter? _ordersProcessed;
private OrderMetrics()
{
_orderProcessingTime = new EventCounter("order-processing-time", this)
{
DisplayName = "Order Processing Time (ms)",
DisplayUnits = "ms"
};
_ordersProcessed = new IncrementingEventCounter("orders-processed", this)
{
DisplayName = "Orders Processed",
DisplayRateTimeScale = TimeSpan.FromSeconds(1)
};
}
public void RecordProcessingTime(double milliseconds)
=> _orderProcessingTime?.WriteMetric(milliseconds);
public void RecordOrderProcessed()
=> _ordersProcessed?.Increment();
protected override void Dispose(bool disposing)
{
_orderProcessingTime?.Dispose();
_ordersProcessed?.Dispose();
base.Dispose(disposing);
}
}
```text
Monitor custom counters:
```bash
dotnet-counters monitor --process-id <PID> --counters MyApp.Orders
```bash
### Interpreting Counter Data
Use counter values to direct further investigation. See [skill:dotnet-observability] for correlating these runtime metrics with OpenTelemetry traces:
| Symptom | Counter Evidence | Next Step |
|---------|------------------|-----------|
| High CPU usage | `cpu-usage` > 80%, `threadpool-queue-length` low | CPU profiling with dotnet-trace |
| Memory growth | `gc-heap-size` increasing, frequent Gen 2 GC | Memory dump with dotnet-dump |
| Thread starvation | `threadpool-queue-length` growing, `threadpool-thread-count` at max | Check for sync-over-async or blocking calls |
| Request latency | `request-duration` high, `active-requests` normal | Trace individual requests with dotnet-trace |
| GC pauses | High `gen-2-gc-count`, `time-in-gc` > 10% | Allocation profiling with dotnet-trace gc-collect |
### Exporting Counter Data
```bash
# Export to CSV for analysis
dotnet-counters collect --process-id <PID> \
--format csv \
--output counters.csv \
--counters System.Runtime
# Export to JSON for programmatic consumption
dotnet-counters collect --process-id <PID> \
--format json \
--output counters.json
```json
---
## dotnet-trace -- Event Tracing and Flame Graphs
### Overview
`dotnet-trace` captures detailed event traces from a running .NET process. Traces can be analyzed as flame graphs to identify CPU hot paths, or configured for allocation tracking to find GC pressure sources.
### CPU Sampling
CPU sampling records stack frames at a fixed interval to build a statistical profile of where the application spends time:
```bash
# Collect a CPU sampling trace (default profile)
dotnet-trace collect --process-id <PID> --duration 00:00:30
# Collect with the cpu-sampling profile (explicit)
dotnet-trace collect --process-id <PID> \
--profile cpu-sampling \
--output cpu-trace.nettrace
```text
### CPU Sampling vs Instrumentation
| Approach | Overhead | Best For | Tool |
|----------|----------|----------|------|
| CPU sampling | Low (~2-5%) | Finding CPU hot paths in production | dotnet-trace `--profile cpu-sampling` |
| Instrumentation | High (10-50%+) | Exact call counts, method entry/exit timing | Rider/VS profiler, PerfView |
CPU sampling is safe for production use due to low overhead. Use it as the default approach. Reserve instrumentation for development environments where exact call counts matter.
### Flame Graph Generation
Trace files (`.nettrace`) must be converted to a flame graph format for visual analysis:
**Using Speedscope (browser-based, recommended):**
```bash
# Convert to Speedscope format
dotnet-trace convert cpu-trace.nettrace --format Speedscope
# Opens cpu-trace.speedscope.json -- load at https://www.speedscope.app/
```json
**Using PerfView (Windows, deep .NET integration):**
```bash
# Convert to Chromium trace format (also viewable in chrome://tracing)
dotnet-trace convert cpu-trace.nettrace --format Chromium
```bash
### Reading Flame Graphs
Flame graphs display call stacks where:
- **Width** of a frame represents the proportion of total sample time spent in that function (wider = more time)
- **Height** represents call stack depth (taller stacks = deeper call chains)
- **Color** is typically arbitrary (not meaningful) unless the tool uses a specific color scheme
**Analysis workflow:**
1. Look for **wide plateaus** -- functions that consume a large proportion of samples
2. Follow the widest frames **upward** to find which callers contribute the most time
3. Identify **unexpected width** -- framework methods that should be fast appearing wide indicate misuse
4. Compare **before/after** traces to validate optimizations reduced the width of target functions
**Common patterns in .NET flame graphs:**
| Pattern | Likely Cause | Investigation |
|---------|-------------|---------------|
| Wide `System.Linq` frames | LINQ-heavy hot path with delegate overhead | Replace with foreach loops or Span-based processing |
| Wide `JIT_New` / `gc_heap::allocate` | Excessive allocations triggering GC | Allocation profiling with `--profile gc-collect` |
| Wide `Monitor.Enter` / `SpinLock` | Lock contention | Review synchronization strategy |
| Wide `System.Text.RegularExpressions` | Regex backtracking | Use `RegexOptions.NonBacktracking` or compile regex |
| Deep async state machine frames | Async overhead in tight loops | Consider sync path for CPU-bound work |
### Allocation Tracking with gc-collect Profile
The `gc-collect` profile captures allocation events to identify what code paths allocate the most memory:
```bash
# Collect allocation data
dotnet-trace collect --process-id <PID> \
--profile gc-collect \
--duration 00:00:30 \
--output alloc-trace.nettrace
```text
This produces a trace that shows:
- Which methods allocate the most bytes
- Which types are allocated most frequently
- Allocation sizes and the call stacks that trigger them
Correlate allocation data with GC counter evidence from dotnet-counters. If `gen-2-gc-count` is high, the allocation trace shows which code paths produce long-lived objects that survive to Gen 2. See [skill:dotnet-performance-patterns] for zero-allocation patterns to apply once hot allocation sites are identified.
### Custom Trace Providers
Target specific event providers for focused tracing:
```bash
# Trace specific providers with keywords and verbosity
dotnet-trace collect --process-id <PID> \
--providers "Microsoft-Diagnostics-DiagnosticSource:::FilterAndPayloadSpecs=[AS]System.Net.Http"
# Trace EF Core queries (useful with [skill:dotnet-efcore-patterns])
dotnet-trace collect --process-id <PID> \
--providers Microsoft.EntityFrameworkCore
# Trace ASP.NET Core request processing
dotnet-trace collect --process-id <PID> \
--providers Microsoft.AspNetCore
```text
### Trace File Management
| Format | Extension | Viewer | Cross-Platform |
|--------|-----------|--------|----------------|
| NetTrace | `.nettrace` | PerfView, VS, dotnet-trace convert | Yes (capture); Windows (PerfView) |
| Speedscope | `.speedscope.json` | https://www.speedscope.app/ | Yes |
| Chromium | `.chromium.json` | Chrome DevTools (chrome://tracing) | Yes |
---
## dotnet-dump -- Memory Dump Analysis
### Overview
`dotnet-dump` captures and analyzes process memory dumps. Use it to investigate memory leaks, large object heap fragmentation, and object reference chains. Unlike dotnet-trace, dumps capture a point-in-time snapshot of the entire managed heap.
### Capturing Dumps
```bash
# Capture a full heap dump
dotnet-dump collect --process-id <PID> --output app-dump.dmp
# Capture a minimal dump (faster, smaller, but less detail)
dotnet-dump collect --process-id <PID> --type Mini --output app-mini.dmp
```text
**When to capture:**
- Memory usage has grown beyond expected baseline (compare against dotnet-counters `gc-heap-size`)
- Application is approaching OOM conditions
- Suspected memory leak after load testing
- Investigating finalizer queue backlog
### Analyzing Dumps with SOS Commands
Open the dump in the interactive analyzer:
```bash
dotnet-dump analyze app-dump.dmp
```bash
### !dumpheap -- Heap Object Summary
Lists objects on the managed heap grouped by type, sorted by total size:
```text
> dumpheap -stat
Statistics:
MT Count TotalSize Class Name
00007fff2c6a4320 125 4,000 System.String[]
00007fff2c6a1230 8,432 269,824 System.String
00007fff2c7b5640 2,100 504,000 MyApp.Models.OrderEntity
00007fff2c6a0988 15,230 1,218,400 System.Byte[]
```text
**Analysis approach:**
1. Look for unexpectedly high counts or sizes for application types
2. Compare counts against expected cardinality (e.g., 2,100 OrderEntity objects -- is that expected for current load?)
3. Large `System.Byte[]` counts often indicate unbounded buffering or stream handling issues
Filter by type:
```text
> dumpheap -type MyApp.Models.OrderEntity
> dumpheap -type System.Byte[] -min 85000
```text
The `-min 85000` filter shows Large Object Heap entries (objects >= 85,000 bytes that cause Gen 2 GC pressure).
### !gcroot -- Finding Object Retention
Traces the reference chain from a GC root to a specific object, explaining why it is not collected:
```text
> gcroot 00007fff3c4a2100
HandleTable:
00007fff3c010010 (strong handle)
-> 00007fff3c3a1000 MyApp.Services.CacheService
-> 00007fff3c3a1020 System.Collections.Generic.Dictionary`2
-> 00007fff3c4a2100 MyApp.Models.OrderEntity
## Detailed Examples
See [references/detailed-examples.md](references/detailed-examples.md) for complete code samples and advanced patterns.