Performance Attribution
Pair a measured percentile movement (from Sentry Trace Explorer) with black-box code-diff analysis to produce confidence-rated attributions: what changed across releases, how much it moved, and why.
When To Use
- Explaining a confirmed p75/p95 latency change across releases
- Building a per-release attribution catalogue (change → confidence → metric)
- Auditing whether a "performance initiative" actually moved a metric
- Attributing movement that spans the app repo and
@metamask/*core-package bumps
Do Not Use When
- The metric movement isn't yet confirmed reliable — run query hygiene first (see
sentry-mcp-queries: filter superseded/low-sample releases, normalize, verify stored sample size) - You need proof of causation — this yields likely contributors, not isolated causes (see Limitations)
- Pre-merge perf review of a single PR — there is no production metric to attribute yet
Step 1 — Get the Measurement Right First
Attribution is only as good as the metric. Lock these down before touching code:
- Percentile. p75 = typical user (more stable signal). p95 = slowest 5% — assumed large-wallet/power users, but not cohort-verified (also slow hardware / poor network). Prioritize p95 when the optimization targets data size (memoization, virtualization) that disproportionately helps the tail; trust p75 as the more reliable number.
- Time window. Use the longer (90d) window as primary — it includes traffic from when older releases were actively used, so the population is representative and comparable across releases. A 30d window over-weights residual users still lingering on old versions → inflated baselines and bigger-looking deltas between different populations. Report 90d; cite 30d only as context.
- Version selection. Per minor line, anchor on the highest-sample patch, never the
.0..0releases have 10–100× fewer samples and are rollout-biased. Seesentry-mcp-queries→ Filtering Unreliable Releases and Longer-Range (30D+) Queries and Percentile Fidelity.
Step 2 — Black-Box Code Analysis
Assess impact on code content + execution frequency alone. Deliberately ignore commit messages, PR titles/descriptions, claimed impact, and epic/initiative goals — they bias the read. Base it on: diff content, file location (→ execution frequency), algorithmic complexity, and memoization patterns.
Hot-path categories — a change here can move a render metric:
| Category | Why it matters |
|---|---|
| Build config | Build-time transforms (e.g. React Compiler) apply broadly |
| Selectors | Run on every state change — hottest path |
| Hooks | Affect component lifecycle / re-render frequency |
| Components | Virtualization & render patterns |
| Dependencies | Version bumps change runtime behavior |
Pattern catalogue — what to grep for:
| Pattern | Signal | Confidence |
|---|---|---|
createDeepEqualSelector → createSelector + EMPTY_ARRAY sentinel |
Removes per-change deep compares | HIGH if foundational selector |
Identity-function selector (foo => foo) → real transform |
Broken memoization fixed | HIGH if many consumers |
In-place mutation .sort()/.reverse()/.splice() → spread copy |
Mutation had broken all downstream memoization | HIGH |
| Build-plugin addition with broad scope | Build-time optimization | HIGH if scope = all ui/ |
| O(n) string parse → O(1) lookup | Algorithmic reduction | MEDIUM — depends on call frequency |
Step 3 — Score Confidence
- Mechanism — how does this reduce work? (fewer re-renders / less allocation / better caching)
- Frequency — is the path hot? (selector per state change = hot)
- Scope — how many components/files does it touch?
- Match — does it target what the metric measures?
| Confidence | Criteria |
|---|---|
| HIGH | Clear mechanism + hot path + timing matches the metric move |
| MEDIUM | Mechanism clear, frequency or scope uncertain |
| LOW | Indirect or infrastructure-only |
Step 4 — Don't Forget Core Packages
App-repo diffs miss work shipped as @metamask/* version bumps — it surfaces only as a package.json change. For each bump between releases, read the package CHANGELOG "Changed"/"Fixed" sections for state-size reduction, caching, fewer RPC calls, batching, or data-structure/field deprecations. (Commands and the packages to watch live in the repo file.)
Reading / Writing an Attribution Catalogue
- Release-header totals = the measured improvement for the whole release
- Table rows = likely contributors, not isolated causes
- "High confidence" = mechanism + timing + population align
- Always keep an Unattributed section for movement no change explains
- Flag high-variance metrics (a noisy confirmation-popup p95) as inconclusive, not as wins
Limitations
- Correlation, not causation — change + improvement in the same release does not prove the change caused it
- Release totals, not isolated impact — a "-44%" reflects the entire release, not one change
- Production variance — user hardware and network are uncontrolled, and each release's user mix differs (early updaters on a new release, lingering users on an old one)
- Code analysis, not runtime profiling — based on structure, not measured execution paths
- p95 cohort is assumed, not verified — no power-user segmentation
- Window choice changes the baseline — always state which window a number came from
For more precise attribution: per-optimization feature flags / A-B tests, CI synthetic benchmarks, and verified user-cohort segmentation. Threshold remote feature flags assign each user to a fixed group by hashing a per-user id (the canonical profile ID, or the MetaMetrics ID before a profile exists), so a threshold flag gives a stable A/B split.