# Shortcircuit Engine > Reference for the ShortCircuit XT engine (src/scxt-core/). Covers Engine/Patch/Part/Group/Zone hierarchy, voice management, DSP processors, modulation, group triggers, sample loading, undo, browser, and the test harness. Use before making engine or DSP changes. - Skill: `baconpaul/shortcircuit-engine` (Agent Skill) - Install (CLI): `npx skillmds@latest add baconpaul/shortcircuit-engine` - Raw SKILL.md: https://api.skillmd.com/api/skills/baconpaul/shortcircuit-engine/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: baconpaul (https://skillmd.com/u/baconpaul) - Updated: 2026-09-21 - Page: https://skillmd.com/skills/baconpaul/shortcircuit-engine --- # ShortCircuit XT Engine Reference **Repo:** `surge-synthesizer/shortcircuit-xt`. All paths are relative to the repo root. **Build:** CMake + Ninja, C++20. `$B` below is your configure directory (`build`, `cmake-build-debug`, …). ```bash cmake --build $B --target shortcircuit-products --parallel 8 # plugins + standalone cmake --build $B --target scxt-test --parallel 8 && $B/tests/scxt-test ``` Companion skills: `shortcircuit-streaming` (json + messaging), `shortcircuit-ui` (`src/scxt-plugin/`), `sc-stream-enum` (procedure for streaming an enum). --- ## 1. Hierarchy and ownership ``` Engine engine/engine.h audio processor; owns everything below └─ Patch engine/patch.h 16 parts + the bus matrix └─ Part[0..15] engine/part.h MIDI responder, macros, part FX └─ Group[*] engine/group.h trigger conditions, group modulators, group FX └─ Zone[*] engine/zone.h sample variants, mapping, voice-scope FX └─ Voice voice/voice.h one active playback voice for a zone ``` `Group` and `Zone` share a lot: both have modulators, a processor chain and output settings. That commonality lives in `engine/group_and_zone.h` + `engine/group_and_zone_impl.h` — read those before adding anything that should exist at both levels, because most such things are written once there. ## 2. Constants `configuration.h` is short and is the authority. Read it rather than trusting a table: ```bash grep constexpr src/scxt-core/configuration.h ``` The ones whose meaning is not obvious from the name: | Constant | Note | |---|---| | `blockSize` (16) | Every DSP loop is written against this. Not runtime-configurable. | | `maxBusses` | `1 + numParts + numAux` — main, one per part, four aux. Bus indices are this flat space, not three separate ranges. | | `egsPerZone` (5) | AEG plus 4 modulation envelopes. `egsPerGroup` is 2, so group and zone envelope code is not symmetric. | | `maxGeneratorsPerVoice` (64) | One voice can play many samples at once (unison, multi-variant). | | `maxRoundRobinSets` / `maxRoundRobinGroupsPerSet` / `maxRoundRobinOrdinal` | Round-robin group triggering; see §6. | | `relativeSentinel` | Marker string standing in for "path relative to the patch" in saved files. | `configuration.h` also holds `namespace scxt::log` — a block of `static constexpr bool` flags (`voiceLifecycle`, `sampleLoadAndPurge`, `groupTrigggers`, `undoRedo`, …) used with `SCLOG_IF(flag, msg)`. Flip one on locally to trace a subsystem. **Leave every flag false in anything you commit.** ## 3. The audio thread `Engine::processAudio()` runs once per 16-sample block: 1. Drain the serialization→audio ring buffer and run the queued lambdas. 2. Update transport phasors. 3. `Patch::process()` — parts → groups → zones → voices, accumulating into the bus matrix, then part FX, then aux sends, then main. 4. Accumulate the preview voice if one is active. 5. Update bus VU levels and, at roughly 30 Hz, `SharedUIMemoryState`. **The rules, without exception:** no allocation, no locks, no `std::string`, no file or console I/O on this thread. Sample data is read-only here; it is loaded on the serialization thread and never mutated while voices might be reading it. `engine/memory_pool.h` exists for the cases that genuinely need scratch memory on the audio thread — it hands out pre-allocated blocks rather than allocating. ### Voice lifecycle Note-on → `Engine::processNoteOnEvent()` → `findZone()` → the voice manager decides allocation → `engine_voice_responder.cpp` creates the voices. Voices are **placement-new'd into one pre-allocated block**: `voiceInPlaceBuffer` holds the storage and `std::array voices` points into it. Nothing is heap-allocated when a note arrives. Do not add a member to `Voice` that allocates in its constructor. Note-off releases; termination fades over a few blocks and then reclaims the slot. Zones also keep `voiceWeakPointers` so they can find their live voices without owning them. Voice allocation policy — stealing, mono/legato, polyphony groups — is **not** in this repo. It is `sst-voicemanager`, wired up through the responder in `engine/engine_voice_responder.cpp`. See the `sst-voicemanager` skill. ### findZone `findZone()` walks part → group → zone and returns every zone that should sound: 1. Active, unmuted parts that respond on the event's MIDI channel. 2. Per group: apply transpose, then evaluate `GroupTriggerConditions` (§6). 3. Per zone: keyboard range and velocity range. A single note routinely matches several zones across several groups. ## 4. Voice structure `voice/voice.h`. A voice holds: - Up to `maxGeneratorsPerVoice` generator slots — `GeneratorState` (position, bounds, loop), `GeneratorIO` (buffers) and a `GeneratorFPtr` chosen per slot. - Four processor slots (zone-scope, one instance per voice). - The AEG plus four modulation envelopes, and the voice modulation matrix. - `output[2][blockSize << 2]` — oversized because groups can run 2× oversampled. `Voice::process()` runs envelopes → modulation matrix → pitch → generator ratios → generator → pan/amp → processor chain → accumulate. ### Generators `dsp/generator.h`. Position is fixed-point: `samplePos` plus `sampleSubPos`, with `ratio` in Q24 (`1<<24` == 1.0×). The generator is picked as a function pointer over the full cross product of stereo/mono, float/int16, and loop configuration: ```cpp GeneratorFPtr GetFPtrGeneratorSample(bool isStereo, bool isFloat, bool loopActive, bool loopForward, bool loopWhileGated); ``` Adding a playback behaviour usually means adding to that cross product, which is why the file is generated-looking. Interpolation is `Sinc` (default), `Linear`, `ZOHAA` or `ZeroOrderHold`. Loop crossfade, reverse and alternate-direction playback interact subtly with the loop bounds. `tests/` has direct generator coverage — use it, this is easy to get wrong. ## 5. Processors and buses `dsp/processor/` — filters, EQ, delay/reverb, distortion, modulation effects, utility. Storage is uniform: ```cpp struct ProcessorStorage { ProcessorType type; float floatParams[maxProcessorFloatParams]; // 9 int32_t intParams[maxProcessorIntParams]; // 5 uint32_t unstreamed; }; ``` Every processor lives inside that fixed budget, which is why parameters are indexed rather than named. The UI reads names and ranges from the processor's own metadata — see `shortcircuit-ui`. Four slots exist at each of four scopes: zone (per voice), group (shared by the group's voices), part, and bus. `engine/bus.h` + `engine/bus_effect.h` cover the bus-level chain, sends and VU. Groups can run 2× oversampled via `Group::outputInfo.oversample`; that is why voice output buffers are oversized. ## 6. Group triggers `engine/group_triggers.h` — how a group decides whether to sound at all. This is a real subsystem and easy to miss. `GroupTriggerID` covers keyswitches (latch and momentary), program change, pitch bend, round robin (cycle / random / shuffle), plus a contiguous block of macro IDs and a block of 128 MIDI CC IDs at the end. `MACRO` and `MIDICC` are range bases, not single values — `MIDICC + cc` is the ID for a given CC, which is why the comments insist they stay last. `GroupTriggerConditions` holds `triggerConditionsPerGroup` conditions plus a `Conjunction` between them. Round robin has three independent kinds; `roundRobinKindIndex()` maps an ID to 0/1/2 and each kind gets its own set space. Cycle set 1 and random set 1 are unrelated round robins, not one set two groups disagree about. A note's set membership is a `roundRobinMask_t` — one bit per set, one mask per kind. `VoiceCreationMode` on the conditions decides when the group makes voices: `ON_NOTE_ON` (everything historically) or `ON_NOTE_OFF`, where the press is only remembered and the group sounds on key release at the press velocity. Latch state and round-robin position are runtime state living in `GroupTriggerInstrumentState`, deliberately not streamed. ## 7. Modulation Both `Group` and `Voice` get their modulators from `HasModulators` in `engine/group_and_zone.h`: LFO storages (step, curve, env), ADSRs, phasors, randoms, envelope followers, and a matrix. The counts differ between group and zone — read `configuration.h`. The matrix is source → target with four-character identifiers: ```cpp struct SourceIdentifier { char gid[4]; uint32_t index; }; // 'lfo ', 'envg', 'phsr', 'rand', … struct TargetIdentifier { char gid[4]; char id[4]; uint32_t index; }; // 'proc'/'cut ', 'zout'/'pan ', … ``` `modulation/` holds the implementations, `engine/*` the storage. Routing rows are streamed; the resolved pointer maps are rebuilt on unstream, not saved. The matrix is on the audio-thread hot path and has been made allocation-free — `describeValue`-style helpers and anything returning a `std::string` must not be called from `process()`. ## 8. Samples `sample/sample.h` — a `Sample` owns up to two channel buffers (`BD_I16` or `BD_F32`), a sample rate, and a `meta` block carrying whatever the source format told us: root key, key and velocity ranges, loop points, slices, beat count. Each has a `*_present` flag, because "the file said nothing" and "the file said zero" are different. `compoundSourceDetails` identifies a sub-sample inside a container as `"Preset:Instrument:Region"`. `isMissingPlaceholder` marks a sample that could not be found on load. The patch still loads; `engine/missing_resolution.h` drives the UI flow that asks the user to relocate it. ### Loading `sample/sample_manager.h` owns a `SampleID → shared_ptr` registry behind a recursive mutex. Loads happen on the serialization thread only. `purgeUnreferencedSamples()` drops what nothing points at — note that undo deliberately does *not* purge, because undoing a delete needs the samples still resident. Format support lives in per-format directories under `sample/`: `sf2_support`, `gig_support`, `exs_support`, `sfz_support`, `akai_support`, `multisample_support`, `loaders`. **All of them funnel through `sample/import_support/`** — shared code for building zones and groups out of a parsed instrument. Add per-format quirks in the format directory; add anything about how a parsed instrument becomes zones in `import_support`. ### Zone variants A zone has `maxVariantsPerZone` variants, each a `SingleVariant` with its own sample, start/end, loop configuration, interpolation, pitch offset, amplitude, pan and normalization. `VariantPlaybackMode` picks between them (round robin, random, random-no-repeat). Variants are how round-robin sampling and velocity layering inside one zone are expressed. ## 9. Undo `undo_manager/`. Do not hand-roll undo for a new edit — the machinery is general. `PayloadUndoableItem` (`payload_undoable_items.h`) covers every value-type edit. A spec is about six lines: a name, whether it is zone- or group-scoped, the value type, and how to reach the field. Optional `read` / `write` / `postWrite` / `serialExtra` hooks are detected by named concepts. Around twenty specs cover the whole engine including mixer and processor swaps. Structural edits (delete, restore, reorder, part activation) are in `structure_undoable_items.h` as symmetric do/undo pairs. `UndoManager` (`undo.h`) additionally supports **gestures**. Handlers always push a discrete undo step; a gesture tag (`Spec::name() + "/" + index`) opened by a begin-edit message folds an entire drag into one entry. `storeUndoStep` closes any open gesture, so a gesture that is never explicitly closed still terminates at the next unrelated push. Use `pushUndo(engine, …)` / `pushUndoTagged(...)` rather than touching the stacks. Loads and imports are undoable, not stack-clearing: they push a full streamed snapshot of the engine or the part. Only DAW state unstream and engine reset clear the stacks. Sharp edges worth knowing: - In `CLIENT_TO_SERIAL` macro bodies, wrap multi-template-argument calls in an extra set of parentheses — the preprocessor splits on the comma in ``. - Prefer named concepts to inline `requires{…}` referencing generic-lambda parameters; clang mis-evaluates the latter during substitution. ## 10. Browser `browser/browser.h` classifies files (`isLoadableSingleSample`, `isLoadableMultiSample`) and expands containers into their instrument list via `expandForBrowser()`. Anything multi-sample must be expanded before it means anything to the user. `browser/browser_db.h` is a SQLite index with a background `WriterWorker` for scanning. `numberOfJobsOutstanding()` is how the UI knows a scan is still running. ## 11. Patch IO `patch_io/patch_io.h`. Save styles: `NO_SAMPLES` (external references), `WITH_COLLECTED_SAMPLES` (copy alongside, relative paths), `AS_MONOLITH` (samples embedded in the RIFF container), `AS_SFZ` (text export). `saveMulti` / `loadMulti` handle whole engines; `savePart` / `loadPartInto` a single part. Streaming context is a thread-local set by a guard: ```cpp engine::Engine::StreamGuard sg(engine::Engine::StreamReason::FOR_PART); ``` `StreamReason` is `IN_PROCESS`, `FOR_MULTI`, `FOR_PART`, `FOR_DAW`. Streaming code reads it through `SC_STREAMING_FOR_DAW` and friends to decide what to include. The current streaming version and the backward-compatibility helpers are covered in `shortcircuit-streaming` — read the constant from `configuration.h`, never from memory. ## 12. Shared UI memory `Engine::SharedUIMemoryState` is a block of atomics the UI reads directly, bypassing the message queue entirely: per-bus VU levels, voice count, a per-voice display record (part / group / zone / sample, sample position, note, gated) and transport state. This exists because VU meters and waveform playback cursors need to be current, and pushing them through the serialization thread at frame rate would swamp it. It is the *only* sanctioned path around the messaging layer. Everything else goes through `s2c`. ## 13. IDs `EngineID`, `PatchID`, `PartID`, `GroupID`, `ZoneID`, `SampleID` — each with a static `next()`. They exist so that references survive structural change; do not substitute indices. Voices reference their zone by path rather than pointer: ```cpp struct Engine::pathToZone_t { size_t part, group, zone; int16_t channel, key; int32_t noteid; }; ``` ## 14. Threading | Thread | May | Must not | |---|---|---| | Audio | Read engine state, run DSP, drain the s→a ring buffer | Allocate, lock, format strings, do I/O | | Serialization | Allocate, lock `modifyStructureMutex`, load samples, stream JSON | Block for long under the structure lock | | Client/UI | Send `c2s`, receive `s2c`, read `SharedUIMemoryState` | Touch engine memory directly | Structure mutation — adding, deleting or reordering zones and groups — happens on the serialization thread under `modifyStructureMutex`. Some operations additionally need the audio thread stopped; those go through the stop-audio-then-run-on-serial path rather than the plain callback. ## 15. Tests `tests/` builds `scxt-test` (Catch2). There are dozens of files, one per subsystem — `ls tests/` is the index. ```bash cmake --build $B --target scxt-test --parallel 8 $B/tests/scxt-test # everything $B/tests/scxt-test "[undo]" # one tag $B/tests/scxt-test "Some test name" # one case ``` Two ways to get an engine, and picking the right one matters: **`makeEngine()`** from `tests/test_utils.h` — a bare engine driven directly on the test thread. No serialization thread, no client. Right for DSP, voice and mapping tests. It pins tuning to 12-TET so a running MTS-ESP master on the dev machine cannot remap keys out of the zone ranges under test. **`ConsoleHarness`** from `src/clients/console-ui/console_harness.h` — the whole three-thread stack with a headless client attached. Right for anything that has to go through messaging: ```cpp scxt::clients::console_ui::ConsoleHarness th; th.start(); th.sendToSerialization(cmsg::SomeMessage(payload)); th.stepUI(); // pump the round trip; the response is not synchronous ``` `test_utils.h` also has `addBlankZoneToGroup` (zones with no sample still make voices — useful for voice-allocation tests), and `countLiveVoicesInGroup` / `countLiveVoicesForKey`. Prefer the per-key variant: a released voice keeps ringing, so a test that presses several keys must ask about the one it cares about. Helpers go in `test_utils.h` rather than being repeated per file, because the unity build folds every test into one translation unit where duplicate definitions collide. `SCXT_TEST_SOURCE_DIR` and `samplePath()` locate test fixtures under `resources/test_samples`. New test file? Add it to `tests/CMakeLists.txt` — it is an explicit source list. ## 16. Editing under libs/ `libs/` holds submodules of two different kinds, and the distinction matters. **`libs/sst/*`** — `sst-basic-blocks`, `sst-filters`, `sst-effects`, `sst-jucegui`, `sst-voicemanager`, `sst-waveshapers` and friends are Surge Synth Team's own. Editing them is normal; changes flow back upstream. Several have their own skills. **Everything else under `libs/`** — taocpp_json, eurorack, sqlite3, clap, JUCE, libgig — is genuinely third-party. Keep changes minimal and obviously scoped, because every line you touch is a line to re-apply at the next update. The repo-root `.clang-format` applies to the whole tree, so a formatter run inside a vendored third-party file will reformat the entire file and bury a two-line fix in a thousand-line diff. When editing there, make the change without triggering a format pass and check `git diff --stat` shows only what you intended. ## 17. Where things live | Looking for | Directory | |---|---| | Engine, patch, part, group, zone, buses, triggers, macros, undo entry points | `engine/` | | Voice and the voice-manager responder | `voice/`, `engine/engine_voice_responder.cpp` | | Generators, processors, effects | `dsp/` | | LFOs, envelopes, matrices | `modulation/` | | Sample data, format parsers, import | `sample/` | | JSON traits, messaging | `json/`, `messaging/` — see `shortcircuit-streaming` | | Undo items and manager | `undo_manager/` | | File browser and SQLite index | `browser/` | | Save/load | `patch_io/` | | Selection model | `selection/` — also `doc/GroupOrZoneSelection.md` | | Tuning / MTS-ESP | `tuning/` | Longer-form architecture notes live in `doc/` — `CoreArchitecture.md`, `VoiceRouting.md`, `GroupOrZoneSelection.md`, `UIConcepts.md`.