Verify a production change
Merged is not deployed, and deployed is not verified.
Gate on the deploy
Write a command that prints the commit sha one production request is served
by, never a version string, from whatever the service exposes (status
endpoint, build-info route, bootstrap manifest). Where the platform reports
instance revisions directly, prefer that and say so. Then, in a clone of the
service's repository, run:
scripts/deploy-gate.sh --target <merged-sha> --baseline <prior-prod-sha> -- <command>
Raise --samples or --clean-polls for a large fleet, or set --interval and
--max-polls to the platform's rollout pace. Its header documents the gate.
Exit 0: write its last line, "No stale instance observed across N
sampled requests", word for word.
Exit 1, rollout-in-progress: report a rollout in progress with the
on and off counts, not a failure. not-deployed: report that no sample
runs the change.
Exit 2, 3, or 5: fix the invocation, sha, or sampling command, then
rerun.
Exit 4: the baseline already contains the target, so ask which revision
production ran before this deploy.
Limit every claim about what production runs to the samples: never "fully
deployed" or "prod is no longer on the old sha".
If the script cannot run, read its header and apply its rules to the samples
you have.
With every sample on target, still write the sentence above, also when the
ancestry check is pending.
Verify the resulting data
For a one-time migration, backfill, or cleanup. Verification is read-only:
never re-run the task in write mode or repair data. Report discrepancies.
- Use the report-mode log. Its per-row expectations are the only pre-image
available once the writes have happened.
- Check two cohorts: rows that should change and rows that should not.
Only the untouched cohort catches a task that touched too much. Sample each
across the axes that vary, include rows the report treated specially
(partial, skip, conflict), and diff predicted against actual with read-only
queries.
- Check one fleet-wide invariant: the population the task was meant to
eliminate is now empty. That count covers every row the sample missed.
- Reconcile the report's totals against post-change counts.
- Name the evidence: report-vs-actual agreement plus an empty invariant is
strong, but it is not a row-level pre-image. Without one there is no
independent record of what deleted rows contained. Say which you have.
Order between deploys
- When one deploy must precede another, the constraint is usually asymmetric.
- State in the PR body which order is safe, what breaks in the other, and
whether the damage self-heals.
- Verify that against the code on both sides: a client that "tolerates either
order" may still drop a signal under one, for example an unconditional filter
upstream of a conditional relabel.
- A one-time task that deletes rows the old renderer still reads is gated on
both deploys and does not self-heal; say so where the ordering is written
down, because nothing enforces it.
1---2name: verifying-production-changes3description: Verifies a merged change took effect in production by revision ancestry and cohort sampling, and reports a bound over sampled requests rather than "fully deployed". Use when the user says "is it live in prod?", or when asked to confirm or report whether a merged change is live or deployed in production, before answering, even when the build-info, status, or deploy output is already in hand; also when gating on a deploy reaching the fleet, sequencing deploys, or verifying a one-time migration, backfill, or cleanup. Not for CI or build state.4metadata:5 source: https://skillmd.com/skills/bostonaholic/verifying-production-changes6---78# Verify a production change910Merged is not deployed, and deployed is not verified.1112## Gate on the deploy1314Write a command that prints the commit sha one production request is served15by, never a version string, from whatever the service exposes (status16endpoint, build-info route, bootstrap manifest). Where the platform reports17instance revisions directly, prefer that and say so. Then, in a clone of the18service's repository, run:1920```sh21scripts/deploy-gate.sh --target <merged-sha> --baseline <prior-prod-sha> -- <command>22```2324Raise `--samples` or `--clean-polls` for a large fleet, or set `--interval` and25`--max-polls` to the platform's rollout pace. Its header documents the gate.2627- **Exit 0:** write its last line, "No stale instance observed across N28 sampled requests", word for word.29- **Exit 1, `rollout-in-progress`:** report a rollout in progress with the30 on and off counts, not a failure. **`not-deployed`:** report that no sample31 runs the change.32- **Exit 2, 3, or 5:** fix the invocation, sha, or sampling command, then33 rerun.34- **Exit 4:** the baseline already contains the target, so ask which revision35 production ran before this deploy.3637- Limit every claim about what production runs to the samples: never "fully38 deployed" or "prod is no longer on the old sha".39- If the script cannot run, read its header and apply its rules to the samples40 you have.41- With every sample on target, still write the sentence above, also when the42 ancestry check is pending.4344## Verify the resulting data4546For a one-time migration, backfill, or cleanup. Verification is read-only:47never re-run the task in write mode or repair data. Report discrepancies.4849- **Use the report-mode log.** Its per-row expectations are the only pre-image50 available once the writes have happened.51- **Check two cohorts:** rows that should change and rows that should not.52 Only the untouched cohort catches a task that touched too much. Sample each53 across the axes that vary, include rows the report treated specially54 (partial, skip, conflict), and diff predicted against actual with read-only55 queries.56- **Check one fleet-wide invariant:** the population the task was meant to57 eliminate is now empty. That count covers every row the sample missed.58- **Reconcile the report's totals** against post-change counts.59- **Name the evidence:** report-vs-actual agreement plus an empty invariant is60 strong, but it is not a row-level pre-image. Without one there is no61 independent record of what deleted rows contained. Say which you have.6263## Order between deploys6465- When one deploy must precede another, the constraint is usually asymmetric.66- State in the PR body which order is safe, what breaks in the other, and67 whether the damage self-heals.68- Verify that against the code on both sides: a client that "tolerates either69 order" may still drop a signal under one, for example an unconditional filter70 upstream of a conditional relabel.71- A one-time task that deletes rows the old renderer still reads is gated on72 both deploys and does not self-heal; say so where the ordering is written73 down, because nothing enforces it.