Patch To Main

Patch → patchMain ownership rework

Promote the UI's patchCopy to an engine-owned patchMain that is the single main-thread source of truth. The audio thread's patch becomes a pure realtime working copy. The CLAP adapter must never read engine->patch.

This is the standard architecture for SideQuest plugins, not an optional upgrade. New shells come out of the sidequest-setup skill already on it; that skill treats this document as its specification. Use the steps below for a legacy plugin that still owns a patchCopy in the editor.

Migrated and usable as a reference: two-filters (the original port, effect-shaped), freaqy-verb, six-sines (instrument-shaped, the richest — it carries extra DAW session state through SET_AUDIO_DAW_STATE and has the fullest patch_sync test suite). Read the one closest in shape to what you are migrating; verify with grep -rl patchCopy ../<plugin>/src coming back empty.

The model

• Engine owns two patches: Patch patch (audio thread only) and Patch patchMain (main thread only).
• Editor holds Patch &patchMainRef bound to engine->patchMain — not an owned copy.
• All CLAP main-thread calls (stateSave/stateLoad/paramsValue/paramsInfo/ paramsValueToText/paramsTextToValue, and paramsFlush when inactive) read/write patchMain.

Invariant D (establish this first — the whole design rests on it)

Every audio-thread mutation of patch emits an audioToMain message; every UI-originated mutation is written into patchMain by the UI itself.

So draining audioToMain into patchMain fully reconstructs the audio side, and patchMain is always authoritative on the main thread. Verify handleParamValue (the host-automation path) already pushes UPDATE_PARAM unconditionally — it usually does. UI edits already write patchCopy (== patchMain) directly in the data binding's setValueFromGUI, so that half holds for free.

Threading rules

• patchMain: main thread only — all CLAP main-thread calls + the editor idle (the clap-juce-shim runs the JUCE timer on the main thread).
• patch: audio thread only, except inside activate()/deactivate() where the audio thread is guaranteed stopped.

Engine state to add

Patch patch;      // audio-thread working copy
Patch patchMain;  // main-thread source of truth

std::atomic<bool> editorActive{false};               // unified editor-open flag (see below)
std::atomic<bool> mainThreadDrainRequested{false};   // coalesces request_callback
std::atomic<uint32_t> uiForceRebuild{0};             // bump => open editor rebuilds from patchMain

Add #include <atomic>. Do not add a hand-rolled engineActivated flag: the clap helper's isActive() already tells you whether the audio thread is running, and (in a clap::helpers::Plugin subclass) it's the only thing that ever needs the answer. Branch on isActive().

Steps

1. Patch::copyValuesFrom(const Patch &o) — a VALUE copy, never operator=. params/paramMap hold Param* into the owning Patch; assignment would alias them across objects. Copy: every param value + all non-Param streamed DSP state (whatever your additionalToStateImpl writes — e.g. two-filters' filterNodes[i] .model/.config) + name + dirty. Note params is vector<const Param*>, so read from o.params and write via your own paramMap:

   void copyValuesFrom(const Patch &o) {
       for (const auto *p : o.params) paramMap.at(p->meta.id)->value = p->value;
       // + per-app non-Param streamed fields (mirror additionalToStateImpl)
       memcpy(name, o.name, sizeof(name));
       dirty = o.dirty;
   }
   

2. Factor the audioToMain handling so the editor idle and the headless drain share it. The editor idle loop already switches on every audioToMain action; do not duplicate the patch-model half in a second drain. Instead:

   • static bool Engine::handleAudioToMainMessage(Patch &dest, const AudioToMainMsg &m) — applies the patch-model messages (UPDATE_PARAM→dest.paramMap[id]->value, patch-name, dirty, filter/DSP-config) to dest; returns true if handled, false for UI-only messages (VU, LFO step, sample-rate, param-rescan). Make it static — it touches only dest, so the editor (which holds no Engine&, just references to the queues + patchMain) can call Engine::handleAudioToMainMessage(patchCopy, *msg).
   • Engine::drainAudioToMainInto(Patch &dest) (main thread) just loops while (pop) handleAudioToMainMessage(dest, *m); (UI-only messages fall through and are discarded). Used by onMainThread, stateSave, and tests.
   • Editor idle(): if (handleAudioToMainMessage(patchCopy, *aum)) { switch on the handled actions to do the WIDGET refresh only (data is already applied) } else { UI-only cases: VU meter, LFO step display, sample-rate, param-rescan }.

3. Unify the editor-open flag into editorActive. Delete isEditorAttached and the EDITOR_ATTACH_DETATCH message + handler. Audio-thread VU/LFO gates read editorActive.load(std::memory_order_relaxed) (a relaxed atomic-bool load is effectively free, even per-sample).

4. handleParamValue (after pushing UPDATE_PARAM): if no editor is draining, ask the main thread to. Check host first so the flag isn't flipped when host is null (tests):

   if (clapHost && !editorActive.load(std::memory_order_relaxed) &&
       !mainThreadDrainRequested.exchange(true))
       clapHost->request_callback(clapHost);
   

5. onMainThread: if (!editorActive.load()) { mainThreadDrainRequested.store(false); drainAudioToMainInto(patchMain); } (store false before draining so a message arriving mid-drain re-arms a callback). Keep existing rescan handling.

6. activate: patch.copyValuesFrom(patchMain) then setSampleRate(...) (which rebuilds filters/LFOs from patch). Nothing else is needed at activate/deactivate — isActive() is the audio-thread-running flag.

7. stateSave: drop SEND_PREP_FOR_STREAM / readyForStream / the spin-wait. Then:

   if (!engine->editorActive.load()) engine->drainAudioToMainInto(engine->patchMain);
   return patchToOutStream(engine->patchMain, ostream);
   

   (Only drain when the editor is closed — otherwise the idle loop is the queue's consumer and you'd steal its messages. When open, the idle keeps patchMain current.)

8. stateLoad: load into a temp so a parse failure never half-writes patchMain:

   auto tmp = std::make_unique<Patch>();
   if (!inStreamToPatch(istream, *tmp)) return false;
   engine->patchMain.copyValuesFrom(*tmp);
   engine->uiForceRebuild++;                         // open editor rebuilds from patchMain
   if (isActive())                                   // clap helper flag, not a hand-rolled one
       Engine::sendEntirePatchToAudio(engine->patchMain, engine->mainToAudio, _host.host()); // also rescans
   else if (_host.canUseParams())
       _host.paramsRescan(CLAP_PARAM_RESCAN_VALUES | CLAP_PARAM_RESCAN_TEXT);  // else branch: no double
   

   Note sendEntirePatchToAudio — the funnel that pushes a loaded patch into the audio-thread patch via the queue (STOP → SET_PARAM_WITHOUT_NOTIFYING per param → START → POST_LOAD → filter/DSP model msgs) and then rescans the host — belongs on Engine as a static, not on the preset manager: it encodes the engine's mainToAudio protocol, and both the preset manager and the clap adapter call it holding only the queue + host (no Engine instance). The preset manager keeps only file→Patch; its loadX methods call Engine::sendEntirePatchToAudio(patch, mainToAudio, clapHost). Do not thread the patch name through this funnel (see step 16): name/dirty are main-thread-only patch state, so loadX sets them on patch (== patchMain) directly right after fromState and this funnel only moves params + filter config.

9. params* reads (paramsCount/paramsInfo/paramsValue/paramsValueToText/ paramsTextToValue): switch engine->patch → engine->patchMain. But the cookie is the exception — it must keep pointing into patch. patchParamsInfo cookies whatever patch it read, and the host hands that cookie straight back on param events, which process() and the active paramsFlush resolve on the AUDIO thread (paramFromClapEvent trusts a non-null cookie and never consults the patch you pass it). Add a helper on the engine and overwrite the cookie immediately after the info call:

   // engine.h — every clap cookie we hand the host points into the audio-thread `patch`
   void *clapCookieFor(uint32_t paramId)
   {
       auto it = patch.paramMap.find(paramId);
       return it == patch.paramMap.end() ? nullptr : (void *)it->second;
   }
   
   // paramsInfo
   if (!patchParamsInfo(paramIndex, info, engine->patchMain)) return false;
   info->cookie = engine->clapCookieFor(info->id);
   

   If paramsInfo does any further per-param work (six-sines rewrites the displayed name for the primary macro param), read the param from patchMain.params[paramIndex] rather than from info->cookie so it no longer depends on the cookie's identity. Every OTHER place that fills a clap_event_param_value_t.cookie — processUIQueue, paramsFlushMainThread — must use the same helper (processUIQueue already holds a patch param, so it is naturally correct; paramsFlushMainThread is the one that will reach for its patchMain dest by mistake).

10. paramsFlush — branch on isActive() (audio thread when active, main thread when not — the CLAP rule; do not assume it's always main-thread):

    if (isActive()) { for (ev in in) handleEvent(ev); engine->snapAllParams();
                      engine->processUIQueue(out); }          // routes into patch
    else            { engine->paramsFlushMainThread(in, out); } // patchMain only
    

    paramsFlushMainThread: apply incoming CLAP_EVENT_PARAM_VALUE in place to patchMain; then drain mainToAudio, applying SET_PARAM/name/dirty/config to patchMain and emitting param-value + gesture out-events for automatable params — whose cookie is clapCookieFor(paramId), NOT the patchMain dest in hand (step 9). Ignore STOP/START_AUDIO/POST_LOAD/RESCAN (handled at activate or irrelevant while inactive). Never touch patch (the cookie lookup is the one read, and it is pointer-only).

11. Editor: the owned Patch patchCopy; becomes a reference bound to patchMain — rename it Patch &patchMainRef; since it is no longer a copy. Ctor also takes std::atomic<bool> &editorActive and std::atomic<uint32_t> &uiForceRebuild (init the reference members in the init list, patchMainRef first). Set editorActive = true right after idleTimer->startTimer(...); in the dtor set editorActive = false; then clapHost->request_callback(clapHost); before idleTimer->stopTimer(). Remove the EDITOR_ATTACH_DETATCH pushes and any sneakyStartupGrabFrom (the editor now shares patchMain, so it opens already showing correct values).

12. Editor idle rebuild (D4): cache uint32_t lastForceRebuild{uiForceRebuild.load()}. At the top of idle(), if uiForceRebuild.load() != lastForceRebuild, update the cache and call a rebuildFromPatchMain() that refreshes every widget from patchCopy — reuse the existing "a preset was loaded" path (e.g. postPatchChange(patchCopy.name) + dirty-state). This is the only refresh path when a host stateLoad arrives while the engine is deactivated (no audio thread to push a full refresh).

13. Update the CLAP createEditor call site to pass engine->patchMain, engine->editorActive, engine->uiForceRebuild.

14. Retire the full-refresh push. Once the editor shares patchMain, the old pushFullUIRefresh (echoing every value/filter-config/name/dirty back through the queue) is dead: the editor renders from patchMainRef on open, and the load paths refresh via postPatchChange / uiForceRebuild. Delete pushFullUIRefresh, the doFullRefresh flag, and its postLoad trigger. The ONLY thing it carried that isn't patch state is the engine's sample rate (footer readout). Keep a lightweight request for exactly that: rename the editor's REQUEST_REFRESH message to REQUEST_NON_PATCH_STATE, and have the engine answer it by pushing only SEND_SAMPLE_RATE (still also pushed by setSampleRate for live rate changes). Drop any duplicate refresh push in the editor ctor. If your plugin has other engine-only (non-patch) UI state, that message is where it belongs.

15. A param rescan is a direct main-thread call — do not round-trip it. Telling the host "re-read the values" (clap_host_params->rescan(VALUES|TEXT)) is a main-thread op, and its only trigger is a bulk out-of-band load (state load / preset load) which already happens on the main thread with patchMain updated first (the host reads values from patchMain). So call rescan straight from the load funnel (sendEntirePatchToAudio, which holds the host) and from stateLoad's deactivated branch. Delete the queue apparatus that bounced it through the audio thread — a SEND_REQUEST_RESCAN mainToAudio message that set an onMainRescanParams atomic AND pushed a DO_PARAM_RESCAN audioToMain message, so the rescan happened twice (once in onMainThread, once in the editor idle). All of that collapses to one direct call. rescan(VALUES|TEXT) is legal while active; only RESCAN_ALL/info/count require an inactive plugin.

16. Patch name + dirty are main-thread-only state — the editor owns them; kill the echo AND the forward message. The audio-thread patch never reads its own name/dirty (they aren't streamed from patch — stateSave reads patchMain; copyValuesFrom overwrites them at activate). So the whole SEND_PATCH_NAME→SET_PATCH_NAME / SET_PARAM-sets-dirty→ SET_PATCH_DIRTY_STATE/SEND_PATCH_IS_CLEAN round-trip is dead weight. Remove both the audio→main echoes (SET_PATCH_NAME, SET_PATCH_DIRTY_STATE) and the main→audio forwards (SEND_PATCH_NAME, SEND_PATCH_IS_CLEAN) plus every producer/handler, and have the editor own the two directly:

    • dirty: a markPatchDirty() on the editor — if (patchMainRef.dirty) return; patchMainRef.dirty = true; presetDataBinding->setDirtyState(true); presetButton->repaint(); — called at each UI edit site that pushes SET_PARAM (both data-binding setValueFromGUIs, setAndSendParamValue when notifyAudio, swapFilters). This preserves the exact old trigger (dirty flips on a user param edit; host automation via handleParamValue still never dirties). Make postPatchChange set setDirtyState(patchMainRef.dirty) so the view always mirrors the model; clear patchMain.dirty in the load funnel and at save.
    • name: setPatchNameTo already writes patchMainRef.name + refreshes the display directly; just drop its SEND_PATCH_NAME push. Preset loads set patch.name (== patchMain) directly in loadX (the name isn't streamed, so something must, and the preset manager is the natural owner). Host stateLoad leaves the name as-is. After this, handleAudioToMainMessage handles only UPDATE_PARAM, the editor idle's "handled" branch collapses to the one widget refresh, and the dead struct fields (patchNamePointer, the audio→main uintValues, the main→audio uiManagedPointer) come out too.

The payoff: sharing patchMain makes a class of round-trips vestigial — sweep for them

Steps 14–16 are not one-off cleanups; they are instances of a single principle. Before this rework, the editor owned a separate patch, so a lot of code existed purely to ferry patch state main → audio → main: the UI asked the audio thread to send state back, or a main-thread event (load, rescan) was bounced through the audio thread so the queue-driven editor would see it. Once the editor and the main-thread source of truth are the same object (patchMain), every one of those ferries is dead weight — the editor already has the state, and main-thread work can happen directly on the main thread.

So after the core rework, actively hunt for and delete round-trips. The test for each candidate: "Does this exist only to move patch state the editor can now read directly, or to bounce a main-thread action through the audio thread and back?" If yes, delete it and do the work directly. In two-filters this swept out, in order:

• sneakyStartupGrabFrom — the editor no longer needs to grab a startup snapshot (step 11).
• pushFullUIRefresh + doFullRefresh — the full value/name/dirty/config echo (step 14).
• the SEND_REQUEST_RESCAN / DO_PARAM_RESCAN / onMainRescanParams rescan bounce (step 15).
• the name + dirty echoes and forwards (SET_PATCH_NAME/SET_PATCH_DIRTY_STATE/SEND_PATCH_NAME/ SEND_PATCH_IS_CLEAN) — name/dirty are patch state the editor sets on patchMain directly (step 16).
• a redundant engineActivated flag — isActive() already answers "is the audio thread running?"

The survivors are messages carrying state that genuinely is NOT in the patch or that genuinely crosses threads: audio→main VU/LFO telemetry; the audioToMain param echo that keeps patchMain current when a host automates while the editor is closed; and the REQUEST_NON_PATCH_STATE → SEND_SAMPLE_RATE round-trip (step 14). Keep that idiom even though two-filters' only rider today is the sample rate — it is the channel for engine-owned, non-patch UI state, and richer plugins (six-sines, shortcircuit) push more through it (extra DAW/engine state, meters that aren't in the patch). It is deliberately NOT collapsed into a direct reference. Everything that touches the queues to move patch state, though, should be suspect.

Pitfalls (each has bitten this pattern)

• operator= on Patch aliases paramMap/params across objects → corruption. Value-copy only. Add a pointer-identity test: after copyValuesFrom, dst.paramMap[id] != src.paramMap[id].
• The clap param cookie is the one main-thread read that must NOT come from patchMain (step 9). Getting this wrong is silent and total: the host's automation writes patchMain from the audio thread and the engine never hears it. Worse than a stale value — handleParamValue does lag.setTarget + paramLagSet.addToActive(p), so the audio-thread lag set ends up ticking a patchMain param every block, racing the editor and stateSave. Sweep every .cookie = and every patchParamsInfo call site after the rework.
• params is vector<const Param*> — you cannot write through it; write via paramMap.
• paramsFlush is audio-thread when ACTIVE, main-thread when INACTIVE. Branch on isActive().
• Two consumers on the SPSC audioToMain queue steal from each other. Gate strictly: editor idle when editorActive, else onMainThread; stateSave drains only when closed.
• Deactivated stateLoad with the editor open has no audio thread to refresh the UI — that is what uiForceRebuild is for.
• Same trap in paramsFlushMainThread: when INACTIVE, a host param change lands there, not on the audio thread, so nothing pushes UPDATE_PARAM to refresh an open editor. If any incoming CLAP_EVENT_PARAM_VALUE was applied to patchMain, uiForceRebuild++ (same out-of-band-write mechanism as stateLoad). Without this the widgets silently lag the model while inactive.
• stateSave with the editor OPEN reads patchMain without draining audioToMain (the idle owns the queue), so it can lag the audio thread by up to one idle tick during an automation burst. This is a window of inconsistency, not a race, and it's acceptable — just know it's there. (When closed, stateSave drains first, so it's exact.)
• The "non-Param streamed state" in copyValuesFrom is app-specific. Mirror exactly what additionalToStateImpl serializes (two-filters: filter model/config; other plugins differ).

Tests (link the impl target; no CLAP host needed)

six-sines/tests/patch_sync.cpp is a worked version of everything below — UI edit reaching audio, automation draining back into patchMain, paramsFlushMainThread forcing a UI rebuild, DAW state round-trip, and a clap-cookie case. Port from it rather than writing these from scratch.

Construct Engine/Patch directly. handleParamValue only calls request_callback when clapHost is set (null in tests). Add to the test executable the impl's PRIVATE header deps that engine.h transitively needs — for two-filters: simde sst-cpputils sst-filters sst-filters-extras sst-plugininfra::patchbase sst-plugininfra::filesystem sst-plugininfra::tinyxml (the impl links these PRIVATE, so they don't propagate).

1. copyValuesFrom: value equality for every param + non-Param DSP fields + name + dirty, plus the pointer-identity guard.
2. toState/fromState round-trip of values + DSP config (name is not streamed — don't assert it).
3. UI→audio: write patchMain + push BEGIN/SET/END to mainToAudio → processUIQueue(out) with a stub clap_output_events_t that discards → lagHandler.instantlySnap() + snapAllParams() → assert patch.
4. Audio→main: handleParamValue(nullptr, pid, v) + snapAllParams() → assert patch; then drainAudioToMainInto(patchMain) → assert patchMain.
5. Drain selectivity: interleave VU/LFO/sample-rate with one UPDATE_PARAM; assert only the param lands and the queue is fully consumed.
6. Cookie routing: patchParamsInfo(idx, &info, patchMain) then info.cookie = clapCookieFor(info.id); assert the cookie equals patch.paramMap.at(pid) and differs from patchMain.paramMap.at(pid). Then build a clap_event_param_value_t carrying that cookie, run it through paramFromClapEvent<Param>(&pevt, patch) + handleParamValue + snapAllParams(), and assert patch moved while patchMain did NOT — then that drainAudioToMainInto catches patchMain up. Mutate the cookie to the patchMain param to confirm the test actually bites.

Follow-up (not done here): a stubbed CLAP test host in sst-clap-helpers to drive the full activate → stateSave/stateLoad path end-to-end.