Shortcircuit Engine

ShortCircuit XT Engine Reference

Repo: surge-synthesizer/shortcircuit-xt. All paths are relative to the repo root. Build: CMake + Ninja, C++20. $B below is your configure directory (build, cmake-build-debug, …).

cmake --build $B --target shortcircuit-products --parallel 8   # plugins + standalone
cmake --build $B --target scxt-test --parallel 8 && $B/tests/scxt-test

Companion skills: shortcircuit-streaming (json + messaging), shortcircuit-ui (src/scxt-plugin/), sc-stream-enum (procedure for streaming an enum).

1. Hierarchy and ownership

Engine                       engine/engine.h    audio processor; owns everything below
 └─ Patch                    engine/patch.h     16 parts + the bus matrix
     └─ Part[0..15]          engine/part.h      MIDI responder, macros, part FX
         └─ Group[*]         engine/group.h     trigger conditions, group modulators, group FX
             └─ Zone[*]      engine/zone.h      sample variants, mapping, voice-scope FX
                 └─ Voice    voice/voice.h      one active playback voice for a zone

Group and Zone share a lot: both have modulators, a processor chain and output settings. That commonality lives in engine/group_and_zone.h + engine/group_and_zone_impl.h — read those before adding anything that should exist at both levels, because most such things are written once there.

2. Constants

configuration.h is short and is the authority. Read it rather than trusting a table:

grep constexpr src/scxt-core/configuration.h

The ones whose meaning is not obvious from the name:

configuration.h also holds namespace scxt::log — a block of static constexpr bool flags (voiceLifecycle, sampleLoadAndPurge, groupTrigggers, undoRedo, …) used with SCLOG_IF(flag, msg). Flip one on locally to trace a subsystem. Leave every flag false in anything you commit.

3. The audio thread

Engine::processAudio() runs once per 16-sample block:

1. Drain the serialization→audio ring buffer and run the queued lambdas.
2. Update transport phasors.
3. Patch::process() — parts → groups → zones → voices, accumulating into the bus matrix, then part FX, then aux sends, then main.
4. Accumulate the preview voice if one is active.
5. Update bus VU levels and, at roughly 30 Hz, SharedUIMemoryState.

The rules, without exception: no allocation, no locks, no std::string, no file or console I/O on this thread. Sample data is read-only here; it is loaded on the serialization thread and never mutated while voices might be reading it.

engine/memory_pool.h exists for the cases that genuinely need scratch memory on the audio thread — it hands out pre-allocated blocks rather than allocating.

Voice lifecycle

Note-on → Engine::processNoteOnEvent() → findZone() → the voice manager decides allocation → engine_voice_responder.cpp creates the voices.

Voices are placement-new'd into one pre-allocated block: voiceInPlaceBuffer holds the storage and std::array<voice::Voice *, maxVoices> voices points into it. Nothing is heap-allocated when a note arrives. Do not add a member to Voice that allocates in its constructor.

Note-off releases; termination fades over a few blocks and then reclaims the slot. Zones also keep voiceWeakPointers so they can find their live voices without owning them.

Voice allocation policy — stealing, mono/legato, polyphony groups — is not in this repo. It is sst-voicemanager, wired up through the responder in engine/engine_voice_responder.cpp. See the sst-voicemanager skill.

findZone

findZone() walks part → group → zone and returns every zone that should sound:

1. Active, unmuted parts that respond on the event's MIDI channel.
2. Per group: apply transpose, then evaluate GroupTriggerConditions (§6).
3. Per zone: keyboard range and velocity range.

A single note routinely matches several zones across several groups.

4. Voice structure

voice/voice.h. A voice holds:

• Up to maxGeneratorsPerVoice generator slots — GeneratorState (position, bounds, loop), GeneratorIO (buffers) and a GeneratorFPtr chosen per slot.
• Four processor slots (zone-scope, one instance per voice).
• The AEG plus four modulation envelopes, and the voice modulation matrix.
• output[2][blockSize << 2] — oversized because groups can run 2× oversampled.

Voice::process() runs envelopes → modulation matrix → pitch → generator ratios → generator → pan/amp → processor chain → accumulate.

Generators

dsp/generator.h. Position is fixed-point: samplePos plus sampleSubPos, with ratio in Q24 (1<<24 == 1.0×). The generator is picked as a function pointer over the full cross product of stereo/mono, float/int16, and loop configuration:

GeneratorFPtr GetFPtrGeneratorSample(bool isStereo, bool isFloat, bool loopActive,
                                     bool loopForward, bool loopWhileGated);

Adding a playback behaviour usually means adding to that cross product, which is why the file is generated-looking. Interpolation is Sinc (default), Linear, ZOHAA or ZeroOrderHold.

Loop crossfade, reverse and alternate-direction playback interact subtly with the loop bounds. tests/ has direct generator coverage — use it, this is easy to get wrong.

5. Processors and buses

dsp/processor/ — filters, EQ, delay/reverb, distortion, modulation effects, utility. Storage is uniform:

struct ProcessorStorage {
    ProcessorType type;
    float floatParams[maxProcessorFloatParams];   // 9
    int32_t intParams[maxProcessorIntParams];     // 5
    uint32_t unstreamed;
};

Every processor lives inside that fixed budget, which is why parameters are indexed rather than named. The UI reads names and ranges from the processor's own metadata — see shortcircuit-ui.

Four slots exist at each of four scopes: zone (per voice), group (shared by the group's voices), part, and bus. engine/bus.h + engine/bus_effect.h cover the bus-level chain, sends and VU.

Groups can run 2× oversampled via Group::outputInfo.oversample; that is why voice output buffers are oversized.

6. Group triggers

engine/group_triggers.h — how a group decides whether to sound at all. This is a real subsystem and easy to miss.

GroupTriggerID covers keyswitches (latch and momentary), program change, pitch bend, round robin (cycle / random / shuffle), plus a contiguous block of macro IDs and a block of 128 MIDI CC IDs at the end. MACRO and MIDICC are range bases, not single values — MIDICC + cc is the ID for a given CC, which is why the comments insist they stay last.

GroupTriggerConditions holds triggerConditionsPerGroup conditions plus a Conjunction between them.

Round robin has three independent kinds; roundRobinKindIndex() maps an ID to 0/1/2 and each kind gets its own set space. Cycle set 1 and random set 1 are unrelated round robins, not one set two groups disagree about. A note's set membership is a roundRobinMask_t — one bit per set, one mask per kind.

VoiceCreationMode on the conditions decides when the group makes voices: ON_NOTE_ON (everything historically) or ON_NOTE_OFF, where the press is only remembered and the group sounds on key release at the press velocity.

Latch state and round-robin position are runtime state living in GroupTriggerInstrumentState, deliberately not streamed.

7. Modulation

Both Group and Voice get their modulators from HasModulators<T, EGCount> in engine/group_and_zone.h: LFO storages (step, curve, env), ADSRs, phasors, randoms, envelope followers, and a matrix. The counts differ between group and zone — read configuration.h.

The matrix is source → target with four-character identifiers:

struct SourceIdentifier { char gid[4]; uint32_t index; };   // 'lfo ', 'envg', 'phsr', 'rand', …
struct TargetIdentifier { char gid[4]; char id[4]; uint32_t index; };  // 'proc'/'cut ', 'zout'/'pan ', …

modulation/ holds the implementations, engine/* the storage. Routing rows are streamed; the resolved pointer maps are rebuilt on unstream, not saved.

The matrix is on the audio-thread hot path and has been made allocation-free — describeValue-style helpers and anything returning a std::string must not be called from process().

8. Samples

sample/sample.h — a Sample owns up to two channel buffers (BD_I16 or BD_F32), a sample rate, and a meta block carrying whatever the source format told us: root key, key and velocity ranges, loop points, slices, beat count. Each has a *_present flag, because "the file said nothing" and "the file said zero" are different.

compoundSourceDetails identifies a sub-sample inside a container as "Preset:Instrument:Region".

isMissingPlaceholder marks a sample that could not be found on load. The patch still loads; engine/missing_resolution.h drives the UI flow that asks the user to relocate it.

Loading

sample/sample_manager.h owns a SampleID → shared_ptr<Sample> registry behind a recursive mutex. Loads happen on the serialization thread only. purgeUnreferencedSamples() drops what nothing points at — note that undo deliberately does not purge, because undoing a delete needs the samples still resident.

Format support lives in per-format directories under sample/: sf2_support, gig_support, exs_support, sfz_support, akai_support, multisample_support, loaders. All of them funnel through sample/import_support/ — shared code for building zones and groups out of a parsed instrument. Add per-format quirks in the format directory; add anything about how a parsed instrument becomes zones in import_support.

Zone variants

A zone has maxVariantsPerZone variants, each a SingleVariant with its own sample, start/end, loop configuration, interpolation, pitch offset, amplitude, pan and normalization. VariantPlaybackMode picks between them (round robin, random, random-no-repeat). Variants are how round-robin sampling and velocity layering inside one zone are expressed.

9. Undo

undo_manager/. Do not hand-roll undo for a new edit — the machinery is general.

PayloadUndoableItem<Spec> (payload_undoable_items.h) covers every value-type edit. A spec is about six lines: a name, whether it is zone- or group-scoped, the value type, and how to reach the field. Optional read / write / postWrite / serialExtra hooks are detected by named concepts. Around twenty specs cover the whole engine including mixer and processor swaps.

Structural edits (delete, restore, reorder, part activation) are in structure_undoable_items.h as symmetric do/undo pairs.

UndoManager (undo.h) additionally supports gestures. Handlers always push a discrete undo step; a gesture tag (Spec::name() + "/" + index) opened by a begin-edit message folds an entire drag into one entry. storeUndoStep closes any open gesture, so a gesture that is never explicitly closed still terminates at the next unrelated push. Use pushUndo<Item>(engine, …) / pushUndoTagged(...) rather than touching the stacks.

Loads and imports are undoable, not stack-clearing: they push a full streamed snapshot of the engine or the part. Only DAW state unstream and engine reset clear the stacks.

Sharp edges worth knowing:

• In CLIENT_TO_SERIAL macro bodies, wrap multi-template-argument calls in an extra set of parentheses — the preprocessor splits on the comma in <A,B>.
• Prefer named concepts to inline requires{…} referencing generic-lambda parameters; clang mis-evaluates the latter during substitution.

10. Browser

browser/browser.h classifies files (isLoadableSingleSample, isLoadableMultiSample) and expands containers into their instrument list via expandForBrowser(). Anything multi-sample must be expanded before it means anything to the user.

browser/browser_db.h is a SQLite index with a background WriterWorker for scanning. numberOfJobsOutstanding() is how the UI knows a scan is still running.

11. Patch IO

patch_io/patch_io.h. Save styles: NO_SAMPLES (external references), WITH_COLLECTED_SAMPLES (copy alongside, relative paths), AS_MONOLITH (samples embedded in the RIFF container), AS_SFZ (text export).

saveMulti / loadMulti handle whole engines; savePart / loadPartInto a single part.

Streaming context is a thread-local set by a guard:

engine::Engine::StreamGuard sg(engine::Engine::StreamReason::FOR_PART);

StreamReason is IN_PROCESS, FOR_MULTI, FOR_PART, FOR_DAW. Streaming code reads it through SC_STREAMING_FOR_DAW and friends to decide what to include. The current streaming version and the backward-compatibility helpers are covered in shortcircuit-streaming — read the constant from configuration.h, never from memory.

12. Shared UI memory

Engine::SharedUIMemoryState is a block of atomics the UI reads directly, bypassing the message queue entirely: per-bus VU levels, voice count, a per-voice display record (part / group / zone / sample, sample position, note, gated) and transport state.

This exists because VU meters and waveform playback cursors need to be current, and pushing them through the serialization thread at frame rate would swamp it. It is the only sanctioned path around the messaging layer. Everything else goes through s2c.

13. IDs

EngineID, PatchID, PartID, GroupID, ZoneID, SampleID — each with a static next(). They exist so that references survive structural change; do not substitute indices.

Voices reference their zone by path rather than pointer:

struct Engine::pathToZone_t { size_t part, group, zone; int16_t channel, key; int32_t noteid; };

14. Threading

Structure mutation — adding, deleting or reordering zones and groups — happens on the serialization thread under modifyStructureMutex. Some operations additionally need the audio thread stopped; those go through the stop-audio-then-run-on-serial path rather than the plain callback.

15. Tests

tests/ builds scxt-test (Catch2). There are dozens of files, one per subsystem — ls tests/ is the index.

cmake --build $B --target scxt-test --parallel 8
$B/tests/scxt-test                       # everything
$B/tests/scxt-test "[undo]"              # one tag
$B/tests/scxt-test "Some test name"      # one case

Two ways to get an engine, and picking the right one matters:

makeEngine() from tests/test_utils.h — a bare engine driven directly on the test thread. No serialization thread, no client. Right for DSP, voice and mapping tests. It pins tuning to 12-TET so a running MTS-ESP master on the dev machine cannot remap keys out of the zone ranges under test.

ConsoleHarness from src/clients/console-ui/console_harness.h — the whole three-thread stack with a headless client attached. Right for anything that has to go through messaging:

scxt::clients::console_ui::ConsoleHarness th;
th.start();
th.sendToSerialization(cmsg::SomeMessage(payload));
th.stepUI();     // pump the round trip; the response is not synchronous

test_utils.h also has addBlankZoneToGroup (zones with no sample still make voices — useful for voice-allocation tests), and countLiveVoicesInGroup / countLiveVoicesForKey. Prefer the per-key variant: a released voice keeps ringing, so a test that presses several keys must ask about the one it cares about.

Helpers go in test_utils.h rather than being repeated per file, because the unity build folds every test into one translation unit where duplicate definitions collide.

SCXT_TEST_SOURCE_DIR and samplePath() locate test fixtures under resources/test_samples.

New test file? Add it to tests/CMakeLists.txt — it is an explicit source list.

16. Editing under libs/

libs/ holds submodules of two different kinds, and the distinction matters.

libs/sst/* — sst-basic-blocks, sst-filters, sst-effects, sst-jucegui, sst-voicemanager, sst-waveshapers and friends are Surge Synth Team's own. Editing them is normal; changes flow back upstream. Several have their own skills.

Everything else under libs/ — taocpp_json, eurorack, sqlite3, clap, JUCE, libgig — is genuinely third-party. Keep changes minimal and obviously scoped, because every line you touch is a line to re-apply at the next update.

The repo-root .clang-format applies to the whole tree, so a formatter run inside a vendored third-party file will reformat the entire file and bury a two-line fix in a thousand-line diff. When editing there, make the change without triggering a format pass and check git diff --stat shows only what you intended.

17. Where things live

Longer-form architecture notes live in doc/ — CoreArchitecture.md, VoiceRouting.md, GroupOrZoneSelection.md, UIConcepts.md.