# Calibrate Engine

> Re-fit or re-anchor the planner's decode/prefill physics (FRAMEWORK_PROFILES, LAYER_OVERHEAD_SCALES, device kernelOverheadScale) against the gold-case corpus, triage outliers and roofline violations, and re-pin the exact-value regression tests. Use when predictions look wrong for a model/runtime/hardware class, when `npm run audit:gold` fails, or after changing any engine formula.

- Skill: `steveseguin/calibrate-engine` (Agent Skill)
- Install (CLI): `npx skillmds@latest add steveseguin/calibrate-engine`
- Raw SKILL.md: https://api.skillmd.com/api/skills/steveseguin/calibrate-engine/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: steveseguin (https://skillmd.com/u/steveseguin)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/steveseguin/calibrate-engine

---


# Calibrate the engine

The engine is a physical model with a small number of fitted constants. Accuracy comes from the
physics being right for each architecture, not from per-model fudge factors. Read the invariants in
`CLAUDE.md` ("The calculation engine") before touching anything.

## The model you are fitting

Per decode pass (all sequences in a batch advance one token):

```
weights   = weight bytes / (BW × bandwidthEfficiency)        gemm = 2N·batch / (TFLOPs × batchedComputeEfficiency(batch))
kv        = KV bytes / (BW × kvReadEfficiency)               attn = 4·heads·dim·depth·batch / (fp32 TFLOPs × DECODE_ATTENTION_EFFICIENCY[kv dtype] / kernelOverheadScale)
overhead  = layers × perLayerOverheadUs × attentionScale × (1 + moeExtra × deviceScale × backendMoeScale) × deviceScale + perTokenOverheadUs × deviceScale
pass      = max(weights, gemm) + max(kv, attn) + overhead + coordination(strategy, interconnect)
```

`DECODE_ATTENTION_EFFICIENCY` (fp16 KV 0.2, q8 0.1, q4 0.06 of the fp32 peak) was fit on the long-context
rows (3060 Ti 65k/98k, V100 46k with f16/q8/q4 KV, 5090 64k); it is what makes deep contexts slow down
more than their KV bytes. Fit it only on rows deeper than ~16k tokens — short rows cannot see it.

All of it lives in `engine.js` (shared by the site and the SDK; `index.html` only holds the UI).

- `FRAMEWORK_PROFILES[runtime]`: `bandwidthEfficiency`, `kvReadEfficiency`, `perLayerOverheadUs`,
  `perTokenOverheadUs`, `prefillEfficiency`, `prefillRampTokens`, `prefillRampFloor`,
  `batchedComputeEfficiency`, `batchRampSequences`, speculation costs (`specDraftStepOverheadUs`,
  `specBatchedDrafting`, `specVerifyReadsKvPerToken`), optional `attentionOverheadScales` /
  `moeOverheadExtra` overrides, and per-backend overrides under `backends` (llama.cpp/Ollama on
  `metal`: `bandwidthEfficiency 0.66`, `moeOverheadScale 6`; Intel `sycl`: `moeOverheadScale 0.25`
  + `moePrefillScale 0.5` on llama.cpp/Ollama, `0.75` + `batchedComputeScale 0.10`
  + `specDraftOverheadScale 3` on vLLM XPU).
  Templates declare `backend` (`metal`, `sycl`); `getBackendEfficiency` merges the overrides.
- `data/lab-evidence.json` + `tests/lab-evidence.test.mjs`: neural.download lab rows (stock /
  lab-baseline / tuned) with shape checks (depth sweep, MTP ladder). Add a row there when the lab
  publishes a new measured series; never put a tuned row into the gold set. `npm test` regenerates
  `data/lab-evidence.js` (what the page loads) and its cache key; the rows surface as the ladder's
  "Nearest measured" (stock/baseline) and "Lab tuned" rungs, in the evidence workspace, and as
  `result.measured` in the SDK — never in calibration.
- `LAYER_OVERHEAD_SCALES.attention[mechanism]` (GDN/KDA hybrids ≈ 2×, MLA 1.25, SSM 1.35) and
  `LAYER_OVERHEAD_SCALES.moeExtra` (routing cost, scaled by the backend).
- `DEVICE_TEMPLATES[*].kernelOverheadScale` (AMD ROCm/Vulkan 1.5, Intel SYCL 2, Apple M5 0.6;
  CUDA/M1–M4 1), `prefillEfficiencyScale` (RDNA4 0.45, 780M 0.2, V100 0.15, M5 0.6, Arc Pro llama.cpp/Ollama 0.6), `backend`.
- `QUANT_FORMATS` bits-per-weight (Q4_K_M 4.9, UD-IQ4_XS 4.0, NVFP4 4.5, AWQ 4.3, Q8_0 8.5 …) and
  `getWeightStorageOverhead` (k-quant scales/metadata 1.16) — the *bytes* side of decode.
- Prefill: `prefillEfficiency × ramp(prompt tokens; floor, rampTokens) × moePrefillFactor
  (sqrt(tokens-per-expert / MOE_PREFILL_TOKENS_PER_EXPERT_REF)) × prefillEfficiencyScale`, with
  `PREFILL_MICRO_BATCH_TOKENS` (512) and `CPU_PREFILL_TFLOPS` (1.2) for offloaded experts.
- Expert offload (llama.cpp/Ollama `--n-cpu-moe`): `describeExpertOffload`, `overflowMode: 'experts'`,
  `EXPERT_OFFLOAD_ROUND_TRIP_US` (150 µs per offloaded layer), `CPU_OFFLOAD_BANDWIDTH_EFFICIENCY`
  (0.5 of DRAM peak for whatever spills to system RAM).
- Speculation: `SPECULATION_METHODS` (acceptance, decay, draft size, KV window, memory), and
  `SPEC_VERIFY_COMPUTE_EFFICIENCY` (0.6). Anchors: llama.cpp MTP ×1.81 (Qwen 3.8 27B, 5090), vLLM
  MTP ×2.56, H100 EAGLE-3 ×2.36 at bs1 / ×1.38 at bs64, draft-model on a 3090 ×0.87.

Each constant has one physical meaning. If you find yourself wanting a constant "for Gemma" or
"for vLLM on 3090s", stop: that is a preset, data, or missing-physics problem.

## Workflow

September 8 review correction: the historical llama-bench `-p` interpretation
in the checklist below is current implementation debt, not verified upstream
semantics. Separate pp/tg tests start tg at `-d` (default zero); `-pg` is the
combined test. See `docs/refresh-review-2026-09-08.md` before recalibrating
these rows. Numerical pins now use a fixed corpus in `tests/fixtures/`;
routine snapshot refreshes must not replace it or repin expected values.

1. **Baseline**: `npm run audit:gold` and `node scripts/fit-decode-constants.mjs` (add `--rows` for
   every row). Note the generic median, within-1.5×, rmsLog, per-group medians, and violations.
2. **Triage the worst rows before fitting anything.** For each outlier, open the raw run (the
   snapshot keeps `command`, `promptTokens`, `outputTokens`, `kvCacheDtype`, `backend`, `splitMode`,
   `peakVramGb`, `source`; the Localmaxxing API row also has `notes`). Known data semantics:
   - Decode depth = recorded `promptTokens` + `outputTokens/2`, not `contextLength` — for llama-bench
     rows too (measured tg falls as 1/p with `-p`; a depth-0 model doubled their error).
   - "Weighted client wall-time throughput" / capacity probes fold a minute of prefill into tok/s —
     excluded by the refresh script when prompt ≥ 32K with no prefill rate; extend that rule if a new
     pattern appears.
   - `-sm tensor` / `-sm row` = tensor parallel in llama.cpp; `-ts 1/1` means 2 of the host's GPUs.
   - `-ctk q8_0` / `--kv-cache-dtype fp8` halve KV bytes; recorded `peakVramGb` below the device pool
     proves a fit even when the uniform byte estimate says overflow (mixed-precision UD quants).
   - FP8 on Ampere, expert-parallel over PCIe, early XPU stacks: genuinely slow, not physics.
   - Speculative runs are excluded from gold on purpose (`isSpeculative` in the refresh script
     reads structured flags, every CLI spelling, notes, and MLX "-mtp" checkpoints); a row that
     beats physics on MLX/oMLX usually is one that slipped through — extend the detector.
   - Prompt-processing rates above the device's dense tensor peak are prompt-cache hits; the refresh
     script nulls them (`plausiblePrefillRate`) so they never calibrate prefill.
   - `--n-cpu-moe N` / `-ncmoe N` rows carry `cpuMoeLayers`; the projection pins that offload. The
     peak-VRAM residency shortcut only applies when the peak is ≥70% of the uniform size estimate; a
     MoE checkpoint far larger than the card with a low `peakVramGb` was auto-fit by llama.cpp (experts
     on the CPU), not a small quant. A recorded `memoryGB` below the template's (3 GB 1060) is the real
     pool.
   A run that beats the **physical roofline** (>1.05×) is always one of these or a preset error.
   Fix the data/preset; never widen a tolerance or lower a ceiling.
3. **Fit**: `node scripts/fit-decode-constants.mjs --grid "FRAMEWORK_PROFILES.llama_cpp.perLayerOverheadUs=35,45,55" "LAYER_OVERHEAD_SCALES.moeExtra=0.4,0.6" …`
   For prefill, fit on the gold rows that carry `prefillTokS` (obs/pred of the system prefill rate;
   group by dense/MoE, prompt-length bucket, and hardware) — the Aug 2026 pass landed llama.cpp at
   `prefillEfficiency 0.7 / prefillRampFloor 0.4 / prefillRampTokens 1536`, median 0.98, 76% within
   1.5×. For a backend/runtime pair that is off on both dense and MoE rows, fit the runtime's
   `backends[backend]` overrides rather than bending the global constants (that is how Metal got
   `0.66 / 6`).
   Rank by rmsLog but choose with judgment: median ≈ 1.0 overall **and** per group (runtime,
   hardware, dense/MoE, depth bucket), no group sacrificed for another, constants that stay
   physically plausible (a per-layer launch floor of 500 µs is not). Prefer changing the constant whose
   physical meaning matches the residual pattern (e.g. MoE rows slow only on Vulkan → backend-scaled
   MoE extra, not a global MoE penalty).
4. **Anchors**: every anchor in `tests/integrity.test.mjs` is a measured number with a source. If a
   fit moves an anchor out of its band, either the fit is wrong or the anchor's band was (re-check the
   measurement). Add an anchor whenever you calibrate on a new measured setup.
5. **Re-pin**: `node scripts/print-regression-pins.mjs` prints the values behind the exact-value tests
   (`four-B70 DeepSeek plan…`, `supplied execution assumptions…`, `AI handoff and Plan JSON…`,
   `projected, optimized, and physical rates stay aligned…`, and the Playwright B70 test). Update the
   pinned ranges only after the physics is final, keep them tight, and say in the commit why they moved.
6. **Verify**: `npm test`, `npm run audit:gold`, `npm run test:playwright`. Then load a few plans in the
   browser: the decode waterfall bands must sum to the per-token total and the "How it scales" curves
   must look monotonic.

## Acceptance envelope (guarded by tests)

Generic engine (no peer correction): median 0.85–1.15, ≥70% within 1.5×, ≥85% within 2×, ≤2% roofline
violations. Leave-one-out calibrated model: median 0.9–1.1, ≥85% within 1.5×, ≥92% within 2×,
optimized-target coverage ≥90%, physical coverage ≥97%. Current state (Aug 2026, 240 rows across 31
device templates): 1.02 / 92% / 96% / 0 (rmsLog 0.37; long-context rows >16k: 0.92, 93% within 1.5×);
prefill (llama.cpp, 129 rows) 0.98 / 76% / 92%. Do not regress these to make a single row fit. `tests/sanity-matrix.test.mjs`
adds wide physical bands across ~500 model × hardware × runtime combinations — if it fails after a
fit, a constant left the plausible range somewhere the gold corpus does not look.

## Things the engine does not model yet (do not fake them with constants)

DeepSeek V4's sparse indexer, prefix caching, chunked-prefill interleaving at high concurrency,
multi-node EXO beyond the coordination term, Apple M5 Neural Accelerator throughput beyond the
derived 120/60 TFLOPS (no vendor figure). Expert offload and speculation *are* modeled now — calibrate
them against paired rows (same rig with/without) rather than absolute rates. If a class of runs is
systematically off because of one of these, add the physics, then re-fit.

## Open calibration items (updated 2026-08-26)

- **DSpark acceptance on large MoE.** `SPECULATION_METHODS.dspark.acceptance`
  (0.78, decay 1.0, 7 tokens → 3.9 tokens/step) is fit on the Qwen 3.8 27B
  DSpark drafter. On DeepSeek V4 Flash the community's own runs gain only
  x1.0-1.1 (4x RTX PRO 6000: 297-346 plain vs 308-310 DSpark; accept_len 4.48
  reported but verify cost eats it) and the lab's 4x B70 REAP run is 80.8 vs
  a 140 stock / 266 optimized projection. Anchor: add those Localmaxxing rows
  (deepseek-ai/DeepSeek-V4-Flash-0731, RTX PRO 6000 x2/x4, vLLM FP8) to the
  gold set and fit a per-model or per-size acceptance, never the ceiling.
- **CUDA fixed floor for 40+ layer MoE.** 4x RTX PRO 6000 DeepSeek V4 Flash
  projects 138 vs 297-346 measured after the TP launch split; the vLLM CUDA
  per-layer overhead is the remaining term. Same rows are the anchor.
- **Arc Pro XMX rates — RESOLVED 2026-08-26.** `float16` held the Xe *vector*
  rate, not the XMX *matrix* rate, understating dense tensor throughput ~4x;
  measured SYCL prefill beat the old ceiling, so those rows were being dropped
  from gold prefill evidence. Xe2 XMX does 2,048 FP16 and 4,096 INT8 ops/clock
  per Xe core, which reproduces Intel's official ARK INT8 exactly and pins the
  peaks at B70 183.5 / B65 98.3 / B60 98.3 / B50 85.2 (`fp8` mirrors `float16`;
  Xe2 has no FP8 datapath). Because the SYCL constants had been fit against the
  understated peak, they were re-fit with it: templates gained
  `prefillEfficiencyScale { llama_cpp: 0.6, ollama: 0.6, default: 1 }`,
  llama.cpp/Ollama `backends.sycl.moePrefillScale 0.5` (a new backend override —
  dense and MoE prefill are not off by the same factor on a stack whose
  grouped-GEMM path is weaker than its dense path), and vLLM XPU
  `batchedComputeScale` 0.40 -> 0.10. Result on the 27 Arc Pro gold rows that
  carry `prefillTokS`: median obs/pred 1.42 -> 0.83, within 1.5x 33% -> 59%,
  vLLM MoE 3.48 -> 0.95; decode is untouched (batch-1 decode is
  bandwidth-bound) and the whole-corpus decode fit is byte-identical at median
  0.99 / 84% / 2 violations. Arc Pro B65 is a real SKU (ARK 245796) and every
  memory, bandwidth, TDP and PCIe value was already correct.
  *Still open here:* the 6 koboldcpp `llama3_8b` B60 rows span 97-826 tok/s
  prefill for the same model on the same card and should be triaged out of the
  corpus; B70 dense prefill rests on 3 rows, one of which sets
  `GGML_SYCL_DISABLE_DNN=1` and so cannot see XMX at all.
- **Research rows flagged `exceedsOptimizedTarget`** in `data/lab-evidence.json`
  (Qwen3.8 INT4 MTP5 TP2 101.17; Muse Glimmer BF16 DFlash draft 100.37) beat
  the engine's optimized target for their stack; they are shown, never used
  as anchors. Revisit once the speculative multi-card XPU path has a second
  measurement.