Dotnet Benchmarkdotnet

dotnet-benchmarkdotnet

Microbenchmarking guidance for .NET using BenchmarkDotNet v0.14+. Covers benchmark class setup, memory and disassembly diagnosers, exporters for CI artifact collection, baseline comparisons, and common pitfalls that invalidate measurements.

Version assumptions: BenchmarkDotNet v0.14+ on .NET 8.0+ baseline. Examples use current stable APIs.

Scope

• Benchmark class setup and configuration
• Memory and disassembly diagnosers
• Exporters for CI artifact collection
• Baseline comparisons and result analysis
• Common pitfalls that invalidate measurements
• Parameterized benchmarks with [Params] and benchmark categories

Out of scope

• Performance architecture patterns (Span, ArrayPool, sealed) -- see [skill:dotnet-performance-patterns]
• Profiling tools (dotnet-counters, dotnet-trace, dotnet-dump) -- see [skill:dotnet-profiling]
• CI benchmark regression detection -- see [skill:dotnet-ci-benchmarking]
• Native AOT compilation and performance -- see [skill:dotnet-native-aot]
• Serialization format performance -- see [skill:dotnet-serialization]
• Architecture patterns (caching, resilience) -- see [skill:dotnet-architecture-patterns]
• GC tuning and memory management -- see [skill:dotnet-gc-memory]

Cross-references: [skill:dotnet-performance-patterns] for zero-allocation patterns measured by benchmarks, [skill:dotnet-csharp-modern-patterns] for Span/Memory syntax foundation, [skill:dotnet-csharp-coding-standards] for sealed class style conventions, [skill:dotnet-native-aot] for AOT performance characteristics and benchmark considerations, [skill:dotnet-serialization] for serialization format performance tradeoffs.

Package Setup

<!-- Benchmarks.csproj -->
<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="BenchmarkDotNet" Version="0.14.*" />
  </ItemGroup>
</Project>

Keep benchmark projects separate from production code. Use a benchmarks/ directory at the solution root.

Benchmark Class Setup

Basic Benchmark with [Benchmark] Attribute

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

[MemoryDiagnoser]
public class StringConcatBenchmarks
{
    private readonly string[] _items = Enumerable.Range(0, 100)
        .Select(i => i.ToString())
        .ToArray();

    [Benchmark(Baseline = true)]
    public string StringConcat()
    {
        var result = string.Empty;
        foreach (var item in _items)
            result += item;
        return result;
    }

    [Benchmark]
    public string StringBuilder()
    {
        var sb = new System.Text.StringBuilder();
        foreach (var item in _items)
            sb.Append(item);
        return sb.ToString();
    }

    [Benchmark]
    public string StringJoin() => string.Join(string.Empty, _items);
}

Running Benchmarks

// Program.cs
using BenchmarkDotNet.Running;

BenchmarkRunner.Run<StringConcatBenchmarks>();

Run in Release mode (mandatory for valid results):

dotnet run -c Release

Parameterized Benchmarks

[MemoryDiagnoser]
public class CollectionBenchmarks
{
    [Params(10, 100, 1000)]
    public int Size { get; set; }

    private int[] _data = null!;

    [GlobalSetup]
    public void Setup()
    {
        _data = Enumerable.Range(0, Size).ToArray();
    }

    [Benchmark(Baseline = true)]
    public int ForLoop()
    {
        var sum = 0;
        for (var i = 0; i < _data.Length; i++)
            sum += _data[i];
        return sum;
    }

    [Benchmark]
    public int LinqSum() => _data.Sum();
}

Memory Diagnosers

MemoryDiagnoser

Tracks GC allocations and collection counts per benchmark invocation. Apply at class level to all benchmarks:

[MemoryDiagnoser]
public class AllocationBenchmarks
{
    [Benchmark]
    public byte[] AllocateArray() => new byte[1024];

    [Benchmark]
    public int UseStackalloc()
    {
        Span<byte> buffer = stackalloc byte[1024];
        buffer[0] = 42;
        return buffer[0];
    }
}

Output columns:

Zero in Allocated column confirms zero-allocation code paths.

DisassemblyDiagnoser

Inspects JIT-compiled assembly to verify optimizations (devirtualization, inlining):

[DisassemblyDiagnoser(maxDepth: 2)]
[MemoryDiagnoser]
public class DevirtualizationBenchmarks
{
    // sealed enables JIT devirtualization -- verify in disassembly output
    // See [skill:dotnet-csharp-coding-standards] for sealed class conventions
    [Benchmark]
    public int SealedCall()
    {
        var obj = new SealedService();
        return obj.Calculate(42);
    }

    [Benchmark]
    public int VirtualCall()
    {
        IService obj = new SealedService();
        return obj.Calculate(42);
    }
}

public interface IService { int Calculate(int x); }
public sealed class SealedService : IService
{
    public int Calculate(int x) => x * 2;
}

Use DisassemblyDiagnoser to verify that sealed classes receive devirtualization from the JIT, confirming the performance rationale documented in [skill:dotnet-csharp-coding-standards].

Exporters for CI Integration

Configuring Exporters

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Exporters;
using BenchmarkDotNet.Exporters.Json;

[MemoryDiagnoser]
[JsonExporterAttribute.Full]
[HtmlExporter]
[MarkdownExporter]
public class CiBenchmarks
{
    [Benchmark]
    public void MyOperation()
    {
        // benchmark code
    }
}

Exporter Output

Custom Config for CI

using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Exporters.Json;
using BenchmarkDotNet.Jobs;

var config = ManualConfig.Create(DefaultConfig.Instance)
    .AddJob(Job.ShortRun)  // fewer iterations for CI speed
    .AddExporter(JsonExporter.Full)
    .WithArtifactsPath("./benchmark-results");

BenchmarkRunner.Run<CiBenchmarks>(config);

GitHub Actions Artifact Upload

- name: Run benchmarks
  run: dotnet run -c Release --project benchmarks/MyBenchmarks.csproj

- name: Upload benchmark results
  uses: actions/upload-artifact@v4
  with:
    name: benchmark-results
    path: benchmarks/BenchmarkDotNet.Artifacts/results/
    retention-days: 30

Baseline Comparison

Setting a Baseline

Mark one benchmark as the baseline for ratio comparison:

[MemoryDiagnoser]
public class SerializationBenchmarks
{
    // Serialization format choice -- see [skill:dotnet-serialization] for API details
    private readonly JsonSerializerOptions _options = new()
    {
        PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
    };

    private readonly WeatherForecast _data = new()
    {
        Date = DateOnly.FromDateTime(DateTime.Now),
        TemperatureC = 25,
        Summary = "Warm"
    };

    [Benchmark(Baseline = true)]
    public string SystemTextJson()
        => System.Text.Json.JsonSerializer.Serialize(_data, _options);

    [Benchmark]
    public byte[] Utf8Serialization()
        => System.Text.Json.JsonSerializer.SerializeToUtf8Bytes(_data, _options);
}

public record WeatherForecast
{
    public DateOnly Date { get; init; }
    public int TemperatureC { get; init; }
    public string? Summary { get; init; }
}

The Ratio column in output shows performance relative to the baseline (1.00). Values below 1.00 indicate faster than baseline; above 1.00 indicate slower.

Benchmark Categories

Group benchmarks with [BenchmarkCategory] and filter at runtime:

[MemoryDiagnoser]
[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)]
public class CategorizedBenchmarks
{
    [Benchmark, BenchmarkCategory("Serialization")]
    public string JsonSerialize() => "...";

    [Benchmark, BenchmarkCategory("Allocation")]
    public byte[] ArrayAlloc() => new byte[1024];
}

Run a specific category:

dotnet run -c Release -- --filter *Serialization*

BenchmarkRunner.Run Patterns

Running Specific Benchmarks

// Run a single benchmark class
BenchmarkRunner.Run<StringConcatBenchmarks>();

// Run all benchmarks in assembly
BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args);

Command-Line Filtering

# Run benchmarks matching a pattern
dotnet run -c Release -- --filter *StringBuilder*

# List all available benchmarks without running
dotnet run -c Release -- --list flat

# Dry run (validates setup without full benchmark)
dotnet run -c Release -- --filter *StringBuilder* --job Dry

AOT Benchmark Considerations

When benchmarking Native AOT scenarios, the JIT diagnosers are not available (there is no JIT). Use wall-clock time and memory comparisons instead. See [skill:dotnet-native-aot] for AOT compilation setup:

[MemoryDiagnoser]
// Do NOT use DisassemblyDiagnoser with AOT -- no JIT to disassemble
public class AotBenchmarks
{
    [Benchmark]
    public string SourceGenSerialize()
        => System.Text.Json.JsonSerializer.Serialize(
            new { Value = 42 },
            AppJsonContext.Default.Options);
}

Common Pitfalls

Dead Code Elimination

The JIT may eliminate benchmark code whose result is unused. Always return or consume the result:

// BAD: JIT may eliminate the entire loop
[Benchmark]
public void DeadCode()
{
    var sum = 0;
    for (var i = 0; i < 1000; i++)
        sum += i;
    // sum is never used -- JIT removes the loop
}

// GOOD: return the value to prevent elimination
[Benchmark]
public int LiveCode()
{
    var sum = 0;
    for (var i = 0; i < 1000; i++)
        sum += i;
    return sum;
}

Measurement Bias

Setup vs Iteration Lifecycle

[MemoryDiagnoser]
public class LifecycleBenchmarks
{
    private byte[] _data = null!;

    [GlobalSetup]    // Runs once before all benchmark iterations
    public void GlobalSetup() => _data = new byte[1024];

    [IterationSetup] // Runs before each benchmark iteration
    public void IterationSetup() => Array.Fill(_data, (byte)0);

    [Benchmark]
    public int Process()
    {
        // uses _data
        return _data.Length;
    }

    [GlobalCleanup]    // Runs once after all iterations
    public void GlobalCleanup() { /* dispose resources */ }
}

Prefer [GlobalSetup] over [IterationSetup] unless the benchmark mutates shared state. [IterationSetup] adds overhead that BenchmarkDotNet excludes from timing, but it still affects GC pressure measurement.

Agent Gotchas

1. Always run benchmarks in Release mode -- dotnet run -c Release. Debug mode disables JIT optimizations and produces meaningless results.
2. Never benchmark in a test project -- xUnit/NUnit test runners interfere with BenchmarkDotNet's measurement harness. Use a standalone console project.
3. Return values from benchmark methods to prevent dead code elimination. The JIT will remove computation whose result is discarded.
4. Do not add manual Thread.Sleep or Task.Delay in benchmarks -- BenchmarkDotNet manages warmup and iteration timing automatically.
5. Use [GlobalSetup] not constructor for initialization -- BenchmarkDotNet creates benchmark instances multiple times during a run; constructor code runs repeatedly.
6. Prefer [Params] over manual loops for parameterized benchmarks. BenchmarkDotNet runs each parameter combination independently with proper statistical analysis.
7. Export JSON for CI -- use [JsonExporterAttribute.Full] to produce machine-readable artifacts for regression detection, not just Markdown.