# Run Integration Tests > Run the New Relic .NET agent integration tests locally. Use whenever you need to run, debug, or reproduce an integration test (host-run IntegrationTests, UnboundedIntegrationTests, or ContainerIntegrationTests), are verifying an agent/wrapper change end to end, or find yourself reaching for `dotnet test` against a test project in this repo. You run these from the CLI yourself -- secrets and infra are already in place. Covers the build-first prerequisite, picking the layer, the run command per layer, and the env gotchas behind confusing failures. - Skill: `newrelic/run-integration-tests` (Agent Skill) - Install (CLI): `npx skillmds@latest add newrelic/run-integration-tests` - Raw SKILL.md: https://api.skillmd.com/api/skills/newrelic/run-integration-tests/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: DevOps & Infra - Author: newrelic (https://skillmd.com/u/newrelic) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/newrelic/run-integration-tests --- # Running integration tests locally Start a real app with the agent attached, exercise it, wait for a harvest, assert on the parsed agent log. Flow: build agent (if changed) -> build test solution (if changed) -> run via the layer's runner -> read results. ## 1. Build (solutions only, never individual projects) **a. Agent -- only if agent/wrapper code changed.** Tests read built home dirs `src/Agent/newrelichome_*`, not source; stale home = #1 false failure. Build `FullAgent.sln` (see **build-dotnet-agent**). Already current for this branch? Skip. **b. Test solution -- once, only if test/app/agent code changed since the last test-solution build.** One build serves all later runs; never rebuild between tests. Building the `.sln` builds the test project, shared `MultiFunctionApplicationHelpers`, and every launched app (the test `.csproj` doesn't reference the apps, but the solution includes all `Applications/*` + `SharedApplications/ConsoleMultiFunctionApplication{FW,Core}`) -- so never build an app by hand. App sourcing at test time: web apps copied from a `Deploy/` folder; FW console from `bin//`; Core console `dotnet publish`ed from source (always current). Use full VS `MSBuild.exe` (not `dotnet build` -- legacy non-SDK web projects have `WebPublish` targets the SDK MSBuild can't build), Debug (fixture looks for apps under `Debug`). These flags mirror CI (`all_solutions.yml`) and are the only proven-good set: `-p:DeployOnBuild=true -p:PublishProfile=LocalDeploy` drives the web publish, and **no `-t:Build`** (the web csproj's `` re-invokes `WebPublish`; `-t:Build` -> `error MSB4006: circular dependency ... "Deploy"`). Slow -- background/tee. Swap the `.sln` per layer. ```bash MSBUILD=$(build/Tools/vswhere.exe -latest -prerelease -products '*' \ -requires Microsoft.Component.MSBuild -find 'MSBuild\**\Bin\MSBuild.exe' | tr -d '\r' | head -1) "$MSBUILD" tests/Agent/IntegrationTests/IntegrationTests.sln \ -restore -m -p:Configuration=Debug -p:DeployOnBuild=true -p:PublishProfile=LocalDeploy ``` ## 2. Pick the layer | Layer | Solution / test project | Notes | |-------|-------------------------|-------| | **Host-run** (default) | `IntegrationTests.sln` / `IntegrationTests.csproj` | Windows + home dirs; real NR staging collectors. FW web-app tests (`RemoteWebApplication`/HostedWebCore) need an **elevated (Administrator)** terminal; console (`ConsoleDynamicMethodFixture`) tests don't. | | **Unbounded** | `UnboundedIntegrationTests.sln` / `.csproj` | Real DBs/brokers (MySQL, Postgres, SQL Server, Mongo, Redis, RabbitMQ, Kafka, Elasticsearch, Couchbase...). | | **Container** | `ContainerIntegrationTests.sln` / `.csproj` | Docker Desktop; Linux-agent coverage. Runner differs -- section 3. | | Performance | `PerformanceTests.sln` | `run-perf-test.py`, not `dotnet test`. Out of scope. | Host-run answers most things. Secrets auto (user secrets on `Shared.csproj`, already set). Unbounded infra is always up (`restart: unless_stopped`) -- no `docker compose up`; check `UnboundedServices` only on a connection failure. ## 3. Run -- runner depends on the layer xUnit v3 projects (OutputType `Exe`, Microsoft.Testing.Platform). **Host-run / Unbounded: run the built exe** `/bin/Debug/net10.0/NewRelic.Agent..exe` (as CI/VS do). **Not `dotnet test`** -- the VSTest bridge is ~3x slower (measured 7 ASB classes: ~5.7 min exe vs ~17 min `dotnet test`). Filters (native xUnit v3; cross-type AND; `*` at start/end): `-namespace ""`, `-class "*Suffix"`/`"FQCN"`, `-method "FQN.Method"`. Dry-run `-list classes` (`/json` to parse) before a slow run. One invocation spans everything in one assembly. Default `-parallel collections`, but effective concurrency ~2 -- fixtures serialize per-app publish/tool-install behind a lock (`DotnetTool`/`RemoteApplication`); don't force wider. Run in a sub-agent (**Sonnet**; **Haiku** for one test) returning only: `TEST EXECUTION SUMMARY` (Total/Failed/Skipped), failing names + asserts, `NR-ERROR`/`NR-FATAL`. Never raw read/tail/wide-grep the agent log -- counts, capped width, `/tmp` distill. ```bash cd tests/Agent/IntegrationTests/UnboundedIntegrationTests/bin/Debug/net10.0 EXE=./NewRelic.Agent.UnboundedIntegrationTests.exe "$EXE" -namespace "NewRelic.Agent.UnboundedIntegrationTests.AzureServiceBus" -class "*CoreLatest" \ -trx "C:/IntegrationTestWorkingDirectory/TestResults/asb.trx" > /tmp/asb.log 2>&1 ``` Exe prints `=== TEST EXECUTION SUMMARY ===` (Total/Errors/Failed/Skipped/Time); `-trx` writes a TRX; the agent-log temp path is printed in output. **Container: use `dotnet test`** (runner is noise -- docker build + harvest waits dominate; ~2m16s `dotnet test` vs ~2m46s exe, same test). Multi-targets net10.0/net11.0 so pass `--framework`; filter by `[Trait]` like `linux_container_tests.yml`. Needs Docker Desktop + Linux home `src/Agent/newrelichome_x64_coreclr_linux` (`libNewRelicProfiler.so`); x64 needs no emulation, arm64 needs QEMU. ```bash dotnet test tests/Agent/IntegrationTests/ContainerIntegrationTests/ContainerIntegrationTests.csproj \ --framework net10.0 --no-build --filter "Architecture=amd64&Distro=Ubuntu" # ... --filter "FullyQualifiedName~UbuntuX64ContainerTest" # one class ``` ## 4. Logging env -- don't touch `NEW_RELIC_LOG_DIRECTORY`/`NEW_RELIC_LOG_LEVEL` are inherited from the user's env -- don't set/override. Change agent behavior via `NewRelicConfigModifier`/`WebConfigModifier`/`fixture.SetEnvironmentVariable`, never ad-hoc XML. ## 5. On failure, check in order 1. **Elevated terminal?** FW web-app tests (RemoteWebApplication/HostedWebCore) need Administrator; else HostedWebCore exits code 2. 2. **Builds current?** `FullAgent.sln` fresh (if agent changed) + test solution built Debug with no app erroring out. (A stale `bin/Release///publish` from `NR_DOTNET_TEST_PREBUILT_APPS=1` can mask source changes -- delete it.) 3. **`AzureFunction` namespace: Core Tools must be >= 4.13.0.** Symptom: `Timed out waiting for Azure function host to become ready`, and the captured `func` stdout carries `Error configuring services in an external startup class ... Could not load file or assembly 'Microsoft.Extensions.Options, Version=10.0.0.0'`. Cause: the isolated `AzureFunctionApplication` extension bundle resolves `Microsoft.Extensions.* 10.0.x` (`Worker.Extensions.Storage.Queues` -> `WebJobs.Extensions.Storage.Queues 5.3.8` -> `Microsoft.Extensions.Hosting >= 10.0.3`) and does not ship those assemblies; the host supplies them. Core Tools retargeted to .NET 10 in 4.13.0 (2026-08-10) -- older hosts are `net8.0` and ship 8.0.0.0. Verify with `func.runtimeconfig.json` (`"tfm": "net10.0"`); winget lags the GitHub release. Select a specific build with `azure_func_exe_path=<...>\func.exe` (CI sets it to the choco path). In-proc classes are unaffected: `AzureFunctionInProcApplication` references the WebJobs extensions directly and builds no bundle. 4. Fixture env vars set? 5. Ports free (localhost)? 6. Agent log: grep `NR-ERROR`/`NR-FATAL`, distilled. 7. App stdout/stderr. Container: Docker up with resources, image built, container stayed up, host<->container reachable.