Golden Source

Golden Value Testing Pattern

This skill adds golden value tests to a C++ project using Catch2. The pattern captures exact floating-point output from a component into hardcoded expected arrays. Tests verify output matches exactly (within a small tolerance). When values need updating, an environment variable switches the test into "print mode" which outputs new arrays to stdout for copy-paste back into the source.

Core Strategy

Two modes, one code path, controlled by an environment variable:

• Verify mode (default): compare got[] against hardcoded expected[] using Approx().margin()
• Print mode (PROJECT_GOLDEN=1): print got[] as a C++ array initializer to stdout

The env var name is derived from the project: e.g. for project MyDSP, use MYDSP_GOLDEN. The user will tell you the project name or preferred env var name.

A note on scope: these tests pin exact numeric output, so they fail on any intentional DSP change as well as every unintentional one. That is the point — but it means a golden test is only worth adding where the numbers are supposed to be stable. Do not put one on something you expect to keep tuning.

Pattern Elements

1. Constants (at file scope)

static constexpr int   GOLDEN_WARMUP = 100;    // samples to discard (let state settle)
static constexpr int   GOLDEN_RECORD = 50;     // samples to capture and compare
static constexpr float GOLDEN_SR     = 48000.f;
static constexpr float GOLDEN_TOL    = 1e-5f;  // absolute tolerance for Approx()
// Add any component-specific constants: frequency, cutoff, etc.

Warmup allows internal DSP state (delay lines, filters, envelopes) to settle before recording begins.

2. Mode check function

static bool goldenPrintMode() { return std::getenv("PROJECT_GOLDEN") != nullptr; }

Replace PROJECT_GOLDEN with the project-appropriate env var.

3. Check-or-print helper

static void goldenCheckOrPrint(const char *label,
                               const std::array<float, GOLDEN_RECORD> &got,
                               const std::array<float, GOLDEN_RECORD> &expected)
{
    if (goldenPrintMode())
    {
        std::printf("// %s\n        {", label);
        for (int i = 0; i < GOLDEN_RECORD; ++i)
        {
            if (i > 0 && i % 5 == 0)
                std::printf("\n         ");
            std::printf("%.9ff%s", got[i], i + 1 < GOLDEN_RECORD ? ", " : "");
        }
        std::printf("};\n");
    }
    else
    {
        for (int i = 0; i < GOLDEN_RECORD; ++i)
        {
            INFO(label << " sample[" << i << "]: got=" << got[i]
                       << " expected=" << expected[i]);
            REQUIRE(got[i] == Approx(expected[i]).margin(GOLDEN_TOL));
        }
    }
}

Key details:

• %.9ff — 9 decimal places captures full float32 precision, trailing f suffix makes it a float literal
• 5 values per line keeps the array readable
• INFO() provides per-sample diagnostics on Catch2 failure (only printed when the assertion fails)
• Approx().margin() uses absolute (not relative) tolerance — correct for DSP values near zero

4. Test case structure

TEST_CASE("MyComponent golden — variant name", "[MyComponent][golden]")
{
    // Initialize component with fixed, deterministic state
    MyComponent comp;
    comp.setSampleRate(GOLDEN_SR);
    // ... other config ...

    // Define a single-sample step function
    auto step = [&]() -> float {
        // drive component, return one output sample
        return comp.process(inputSample);
    };

    // Warmup
    for (int i = 0; i < GOLDEN_WARMUP; ++i)
        step();

    // Record
    std::array<float, GOLDEN_RECORD> got;
    for (int i = 0; i < GOLDEN_RECORD; ++i)
        got[i] = step();

    // Expected — paste output from print mode here
    static const std::array<float, GOLDEN_RECORD> expected{
        0.000000000f, /* ... fill from print mode output ... */
    };

    goldenCheckOrPrint("MyComponent golden — variant name", got, expected);
}

Tag with both a component tag and [golden] so tests can be run selectively:

./my-tests "[golden]"           # all golden tests
./my-tests "[MyComponent]"      # all tests for that component

5. Deterministic input sources

For components that need input (filters, effects), use one of:

LCG noise (flat spectrum, deterministic):

static float lcgSample(uint32_t &rng)
{
    rng = rng * 1664525u + 1013904223u;
    return static_cast<float>(static_cast<int32_t>(rng)) / 2147483648.f;
}
// Seed: uint32_t rng = 0x12345678u;

Fixed-frequency sine (pitched input):

static float sineSample(int &n, float freq, float sr)
{
    return std::sin(2.f * M_PI * freq * (n++) / sr);
}

Use whichever better represents realistic input for the component under test.

Workflow

First time: bootstrap expected values

1. Write the test with a placeholder expected{} (e.g. all zeros)
2. Build and run in print mode:

   PROJECT_GOLDEN=1 ./my-tests "[golden]"
   

3. Copy the printed array back into the expected{} initializer
4. Run in verify mode to confirm it passes:

   ./my-tests "[golden]"
   

After an intentional DSP change

1. Run print mode to regenerate:

   PROJECT_GOLDEN=1 ./my-tests "[golden]" > /tmp/new_golden.txt
   

2. Paste new arrays back into the test source
3. Verify in normal mode

After an unintentional DSP change

Tests fail in verify mode — investigate the regression before updating values.

CMake integration

Typical setup — tests are opt-in:

option(MYPROJECT_BUILD_TESTS "Build test suite" OFF)
if(MYPROJECT_BUILD_TESTS)
    add_subdirectory(tests)
endif()

In tests/CMakeLists.txt:

add_executable(my-tests main.cpp golden_tests.cpp)
target_link_libraries(my-tests PRIVATE Catch2::Catch2WithMain my_dsp_lib)

Build and run:

cmake -DMYPROJECT_BUILD_TESTS=ON -B build .
cmake --build build
./build/my-tests "[golden]"

What to ask the user when applying this skill

1. What is the env var name? (derive from project name, e.g. MYPROJECT_GOLDEN)
2. What directory do the tests live in? (not necessarily src/tests)
3. What component(s) need golden tests? (one test file per component or subsystem is typical)
4. What is the natural input to the component? (none/self-driven, noise, sine, sawtooth, etc.)
5. Is there an existing Catch2 test binary, or does CMake need to be set up?

Reference implementation

The canonical example is surge-synthesizer/OB-Xf, under src/tests/ — osc.cpp for oscillators, filt.cpp for filters, env.cpp for envelopes. Env var OBXF_PRINT_GOLDEN; 50 samples recorded at 1e-5 tolerance.

Note that env.cpp uses a warmup of 0 with a comment saying why: an envelope starts from silence, so there is no state to settle. Pick the warmup from what the component actually does, and leave a comment when it is not the usual 100 — a warmup that is wrong in either direction produces a test that passes while measuring nothing interesting.