EcommerceBenchmarkNexus v3 changelog · Verified against EB code · 2026-08-25
One blocker, one question, and a score shift that would have landed silently
Read against our own adapters, not the changelog alone. Three of my first-pass claims did not survive checking.
What we found 3
BlockingBefore rollout
not_measurable_google_only is unrecognised
An unknown warning code falls through to the renderer, which prints a realmedianDaysActive on a prospect-facing audit.
Our null-score handler checks the warning vocabulary; no match returns null and the guard block is skipped.
Execution continues into the per-metric renderer, which reads rawValue.
That is the one case where Nexus deliberately keeps a real value, so the fall-through renders it graded.
Partial cover exists by value heuristic: suppressed above 270 days Google-inclusive, 180 google-only.
A pinned test asserts the opposite case stays unsuppressed — google-only at a 128 day median keeps the metric.
So a store in that band gets a grade from us on data Nexus has marked unscoreable.
Lower severity
meta_fetch_under_delivered is also unrecognised
It nulls no score, so it cannot fabricate. We want it wired for the signal, not for safety.
It marks inputAdCount as a lower bound rather than a count.
Same thread as the creativesCount cap we already handle.
Missing it costs precision, never correctness.
Question, not a defect
Is the daysActive widening scoped to kingAds[]?
Safe there. But competitor-synthesis carries its own daysActive, and we render one raw.
Our helper treats any non-finite value as unknown; a comment in it already anticipates v3.
evergreen[].daysActive is rendered raw and sorted on, with no guard.
Safe only because the widening does not reach it — an assumption until they confirm.
Every v3 change, against our code
V3 change
What we do today
Verdict
not_measurable_google_only
0 references. Unknown code falls through to the renderer, which reads the kept rawValue.
Blocker
meta_fetch_under_delivered
0 references. Nulls nothing; we lose the lower-bound signal.
Wire it
kingAds[].daysActive → nullable
kingAdsRuntimeDays() returns unknown for any non-finite value.
Safe
snapshot.version → 3
MAX_KNOWN_SCHEMA is 2. We accept, render, and log one warning per worker boot.
Safe, then bump
Google-only returns score: null
Changes our recompute input set and its composite; flips severity for more stores.
Baseline first
Claims I had wrong on the first pass 3
Corrected
“objection is the blocked ask unblocking”
Wrong surface. We already consume an objection dimension.
Ours lives in competitor synthesis and has done for months.
The v3 one is a cohort-distribution dimension — different payload, different surface.
Corrected
“We are safe on daysActive”
True for kingAds[] — and I had grepped only two files.
evergreen[].daysActive is rendered raw and sorted on, with no guard.
Safe only because the widening does not reach it, which is an assumption.
Corrected
“v3 is B-20 being delivered”
The ID resolves, the story exists, and only the title disagrees.
creative.ts:3068 cites dev-handoff B-20 as the ask to null these server-side.
B-20 is a GA4 attribution story. The cite is stale, not wrong-by-fabrication.
Copy blocks
1 · Reply to Valentin M.
One blocker, one question, one thing to coordinate. Nothing asks him to change the release.
## Verified against our code
One blocker, one question, everything else clear.
### BLOCKER: `not_measurable_google_only`
We do not recognise it, and our null-score handler falls
through to the per-metric renderer when a code is unknown.
Since that warning is the one case where you deliberately
KEEP a real `rawValue`, the fall-through renders your real
`medianDaysActive` as a graded finding.
We have a pinned test asserting the opposite case stays
unsuppressed: **google-only at a 128 day median keeps the
metric.** So a store in that band gets a grade from us on
data you have marked unscoreable.
One entry on our side and we ship it before you roll.
Paste into Slack or the ticket
2 · Our work queue, in dependency order
Seven items. Only 1 and 2 have a deadline, and both are ours alone.
## Nexus v3 — EB work queue
| # | Work | Gate | Owner |
|---|---|---|---|
| 1 | Add both warning codes to `resolveNullScoreReason` | Before rollout | EB |
| 2 | Baseline composites so the v3 shift is measurable | Before rollout | EB |
| 3 | Bump `MAX_KNOWN_SCHEMA` 2 → 3 | After validating | EB |
| 4 | Update the limitation in `evalPrompts.ts:249` | When `objection` lands | EB |
| 5 | Consume `platformIds.resolvedBy.first_match` | After rollout | EB |
| 6 | Fix the stale `B-20` cite at `creative.ts:3068` | Anytime | EB |
| 7 | Adopt a Nexus `tier` on batch paths | Blocked on Nexus | Nexus |
For the handoff or a ticket
Worth saying to Vali directly: this changelog did its job. It announces before rollout, names the one non-additive change instead of burying it, and records the v2 failure retroactively so the omission is legible. The single blocker is a code string we now have months earlier than we would have found it by watching a render break — which is what happened with v2.