Spectrumworx Engine

SpectrumWorx — engine and plugin layer

Repo: surge-synthesizer/SpectrumWorx. Paths are relative to the repo root. C++20, CLAP-first, JUCE for the interface only. $B is your CMake configure directory.

cmake --build $B --target spectrumworx_clapfirst_all --parallel   # all four formats
cmake --build $B                                                  # + test binaries + sw-show-ui
ctest --test-dir $B                                               # the CI gate

Useful configure options: -DSW_BUILD_CLAP_ONLY=ON (skips a 54MB SDK fetch), -DSW_BUILD_TESTS=OFF -DSW_BUILD_TOOLS=OFF, -DSW_COPY_AFTER_BUILD=ON, -DSW_WERROR=OFF, -DSW_SANITIZER=<address|thread|realtime|undefined>.

Companion skills: spectrumworx-dsp (effects and the spectral inner loop), spectrumworx-ui (src/gui/).

1. Read doc/tech/ first — it is the authority

This repo has ten maintained technical documents, and they are unusually good: each describes the tree as it is, carries dates and evidence for its claims, and names the tests that pin them. This skill is a router, not a replacement. Where a document covers your question, read it — do not reason from this page or from memory.

doc/tech/README.md indexes them and states the conventions they share. doc/tech/old/ is the port's history — three documents that describe how the tree was read, kept for the reasoning, containing claims the work has since disproved. Do not treat old/ as current.

Work still to do is in GitHub issues, not in these files. Source comments cite them by number (\see issue #12).

2. The layering

sw-dsp      the engine, effects, parameters, preset read/write.  NO JUCE.
 └─ sw-io   external_audio/sample.cpp — the audio file decoder
     └─ sw-gui-resources → sw-gui-widgets → sw-gui        the interface
         └─ sw-impl      the CLAP plugin: spectrumWorxCLAP, host interop

sw-dsp links no JUCE, and that is enforced two ways. checkNoJuceInDSP.cmake (ctest engine-links-no-juce) reads compile_commands.json and fails if any engine translation unit was compiled with -DJUCE_MODULE_AVAILABLE_… — it checks the command, so a JUCE module arriving on the link line fails before anything includes a header. And sw-dsp-tests links only sw-dsp and Catch2, so a test reaching above its layer fails to link.

Preset files are not above the line: presetStorage.cpp uses std::filesystem and <fstream> and lives in sw-dsp. Anything needing juce::File converts at its own edge through io/jucePath.hpp.

Four more source-scan gates run under ctest: odr-header-stays-in-our-sources, asserts-have-no-side-effects, no-juce-file-outside-the-edge, no-colours-outside-the-palette.

3. Threading — the rules that will bite

Read threading_model.md. The short form:

1. The audio thread owns the engine — Program, the module chain, the modules, the LFOs, Engine::Setup — while the plugin is activated.
2. The main thread owns a full copy plus the widgets. It is authoritative for what was asked for; the engine is authoritative for what happened.
3. Three channels, not one: a ToEngine SPSC ring for ordered commands, a ToUI ring for base-value echoes (with Retire on a ring of its own), and a ValueMailbox of atomics for continuously varying modulated values. News that carries no payload — chain changed, timing changed — is a flag, because a flag cannot be dropped on a full ring.
4. The audio thread takes no lock. There is no critical section; it was deleted.
5. A parameter has a base value and a modulated value and they are different things. Base is what the user, the host and a preset write, and what paramsValue, stateSave and a preset read. Modulated is what the LFO writes per block, and it reaches only the mailbox and only for painting.

bool currentThreadMayMutateEngineState()
{ return !engineIsRunning() || Threading::isAudioThread(); }

Threading::isAudioThread() means "is this call inside a CLAP [audio-thread] entry point", not "is this thread the audio thread" — hosts with worker pools deliver successive blocks on different threads. ScopedAudioThreadEntry opens that scope, and a RealtimeSanitizer region with it. Note reset() is [audio-thread] too and runs between blocks; assuming [audio-thread] means "under process()" is what aborted the plugin under the VST3 validator.

Nothing the audio thread touches is destroyed under it. Structural change is publish-and-retire: the main thread builds the replacement, publishes one pointer, and the old one comes back as ToUI::Retire to be deleted on the main thread. Every route that unlinks a module takes a reference first. Use Threading::publish{Slot,ModuleMove,Chain}() — they branch on whether audio is running, which is what keeps headless tests working with no message pump.

The spectral setup is the exception. Changing FFT size, overlap factor or window reallocates the working set, so it is deferred: spectralSetupPending_, then clap_host::request_restart(), applied in deactivate(). That also keeps the CLAP latency contract, since FFT size is the latency.

4. Parameters

Read parameter_system.md. The thing to internalize:

One plugin, a fixed parameter skeleton, slots that change meaning at runtime. Loading a different effect into slot 3 gives the same host-facing parameter a different name, unit, range and meaning — and the effect swap is itself an automatable parameter, so a DAW can automate the meaning of its own parameter list out from under itself.

The skeleton is globals + one selector per module slot + maxNumberOfModules × maxNumberOfParametersPerModule module parameters + the LFO parameters that drive them. src/configuration/constants.hpp and src/core/host_interop/parameters.hpp derive it; the document has the current totals with dates.

SW::ParameterID (src/core/parameterID.hpp) is a packed uint32 union addressing global / module / LFO. Scaling and display are resolved against a nullable Program const * context, because the answer depends on what is loaded.

Each module parameter costs 40 rows — five slots × (itself + seven LFO parameters). That is why the per-module ceiling is a considered number rather than a round one, and why factory.cpp static_asserts that no effect exceeds it.

5. Streaming and presets

Read streaming_format.md. The rule:

Every string that reaches a file comes from a streaming name. Display names are free to change; streaming names are not.

Presets are keyed by name, not index — modules by effect name, parameters by parameter name. That is why the effect list can be reordered without touching a file, and why a preset naming an effect this build lacks degrades rather than corrupting the chain.

Streaming names default to the display name, which is what made the split free: today's display names are the strings inside the shipped presets. LE_SW_EFFECT_STREAMING_NAME exists only for something already renamed after presets had named it. A new effect adds nothing to that table.

Snapshot fixtures pin the format. When you legitimately change it, regenerate deliberately and read the diff: SW_GOLDEN_UPDATE=1, SW_PARAMETER_TABLE_UPDATE=1, SW_STREAMING_NAMES_UPDATE=1.

6. Ports, side chain, MIDI

One channel width, every port. Main in, side chain in and main out all carry 1 or 2; an arrangement where they differ is not representable — Engine::Setup::setNumberOfChannels asserts it and checkChannelConfiguration() accepts only in == out or in == 2*out. A host's mono request never mentions the side chain. See how-mono-ports-work.md.

The side chain is a source, not a topology. Three answers and no fourth — a file, the side input, or nothing. Bus topology is a handshake between plugin, host and track, so a control claiming to decide it would be claiming something it does not get. See sidechain-approach.md.

MIDI arrives on a note port, is parsed once, and lands in two lookalike places: Threading::MIDIMonitor (atomic, crosses to the message thread, drawn by the editor) and Engine::MIDINoteStatus (plain, never leaves the audio thread, read by an effect's setup()). See midi-input.md, including the two events that leave a key stuck if read as a press.

7. Undo

One history per plugin instance, of things the user did in our editor. Host writes are excluded structurally, not by a flag. Three record shapes — value delta, chain edit, state snapshot — because one is not enough. src/undoHistory.{hpp,cpp}; see undo-redo.md, which also names the seam between the plugin's history and the editor that no test reaches.

8. Tests

Two binaries, and the split is load-bearing: sw-dsp-tests links sw-dsp and Catch2 only, which is what proves the engine needs no JUCE; sw-plugin-tests is the CLAP cases, the editor cases and the decoder. ctest runs both. There is no sw-tests.

Both are explicit source lists in tests/CMakeLists.txt — add new files there.

For engine work, the cases that carry the model: core/threadCheckTests.cpp, core/engineOwnershipTests.cpp, core/protocolTests.cpp, clap/hostInteropTests.cpp, clap/threadingTests.cpp, clap/pluginTests.cpp, gui/twoInstanceTests.cpp.

A case that reads one copy cannot see this class of bug. The display, paramsValue, stateSave and the preset writer all answer from the main thread's Program, so a test built out of any of them agrees with an edit the audio thread never received — and asserting that a message was queued only moves the blind spot along. Assert at the far end: hold both copies, check the engine still reads the old value while the command is queued, drive a flush, read again.

Under a sanitizer these same functional cases become the acceptance test, and threadingTests.cpp deliberately creates the overlap so a tsan build has something to report. Two harness notes: REQUIRE from a worker thread races Catch2's own counter, so use the non-asserting processStatus() and assert on the main thread; and flush through ActivePlugin::flush() rather than the extension, since a flush against an active plugin is [audio-thread].

A clean sanitizer run and an inactive one look identical. Check the instrument by reversion — a planted std::malloc in runEngine() is reported; a delete new int is not, because the optimiser removes the pair.

One report is expected and is not a fault: ModuleFactory::create allocating inside process() when a host writes a slot selector. It is recorded as an issue.

9. Sharp edges

• [audio-thread] is a set of entry points, not a thread. reset() is one of them.
• Never destroy on the audio thread. Take a reference, hand it out, retire it.
• paramsFlush is [active ? audio-thread : main-thread]. Branch on it.
• Nothing drains the command queue but process(), paramsFlush() and deactivate(). A queued edit nobody asks for is an edit that never arrives — which is why a preset load calls request_flush at the end.
• A spectral change needs a restart, and no real host has been observed answering request_restart — the test hosts implement it as a no-op observer.
• Assertions compile to nothing under NDEBUG. Anything a shipped build must not do needs something stronger than LE_ASSERT.
• Effect list order is ABI. Append; never insert, reorder or remove.