Backend MVP Guardrails
Overview
Minimize irreversible decisions. Every write must be idempotent, every aggregate must be replayable, and every incident must be attributable with minimal evidence.
When to Use
- MVP backend with single-digit USD/month budget or strict capacity limits
- Fast schema evolution or new data sources with unknown fields
- Third-party backend dependency (e.g., InsForge) with no status page or DB metrics
- Repeated ambiguity about whether failures are vendor or application issues
When NOT to use: throwaway prototypes where data loss and misattribution are acceptable.
Core Pattern (Two Layers)
Layer 1: Principle Guardrails (platform-agnostic)
- Source of truth is immutable or append-only. Avoid online recomputation on read paths.
- Idempotent writes. Deterministic keys + upsert or unique constraint.
- Replayable aggregates. Derived tables can be rebuilt from the source of truth.
- Evidence-first attribution. No structured evidence, no blame, no destructive fix.
- Cost-first queries. Pre-aggregate, cap ranges, enforce limits, avoid full scans.
- Schema evolution is additive. New fields are optional and versioned; unknown fields are rejected by allowlist.
Layer 2: Platform Mapping (InsForge example)
- Fact table: half-hour buckets (e.g.,
vibescore_tracker_hourly)
- Idempotency key:
user_id + device_id + source + model + hour_start
- Aggregates: derived from buckets; do not read raw event tables for dashboards
- Retention: keep aggregates longer; cap any event-level tables
- Backfill: limited window + upsert; must be replayable
- Observability: M1 structured logs (see below)
Responsibility Attribution Protocol (M1)
Required fields: request_id, function, stage, status, latency_ms, error_code, upstream_status, upstream_latency_ms
Attribution rules:
- Missing
upstream_status => UNKNOWN (do not change data semantics)
upstream_status is 5xx/timeout and function status is 5xx => likely vendor/backbone issue
upstream_status is 2xx and function status is 4xx/5xx => likely application validation/logic issue
latency_ms high and upstream_latency_ms low => likely application-side bottleneck
Stop rule: no data rewrite, schema change, or semantic patch without a replay plan and rollback.
Quick Reference
| Guardrail |
Why |
Minimum Implementation |
| Idempotent writes |
Prevent double-counting |
Unique key + upsert |
| Replayable aggregates |
Safe fixes |
Source-of-truth table + backfill job |
| Cost caps |
Fit low budget |
Range limits + pre-aggregates |
| Evidence-first |
Avoid misfix |
M1 structured logs |
| Schema allowlist |
Avoid data bloat |
Reject unknown fields |
Implementation Example (Structured Log)
const start = Date.now();
const requestId = crypto.randomUUID();
const log = (entry) =>
console.log(JSON.stringify({
request_id: requestId,
function: 'example-function',
...entry
}));
try {
const upstreamStart = Date.now();
const res = await fetch(upstreamUrl);
const upstreamLatency = Date.now() - upstreamStart;
log({
stage: 'upstream',
status: res.status,
upstream_status: res.status,
upstream_latency_ms: upstreamLatency,
latency_ms: Date.now() - start,
error_code: res.ok ? null : 'UPSTREAM_ERROR'
});
} catch (err) {
log({
stage: 'exception',
status: 500,
upstream_status: null,
upstream_latency_ms: null,
latency_ms: Date.now() - start,
error_code: 'UPSTREAM_TIMEOUT'
});
throw err;
}
Common Mistakes
- Online aggregation in dashboard endpoints under low budget
- Adding new data sources without updating idempotency keys
- Blame without
upstream_status evidence
- Storing full payloads "just in case" (privacy and cost risk)
- Changing data semantics without replay/backfill plan
Rationalization Table
| Excuse |
Reality |
| "We are a tiny team, logs are overkill" |
Small teams need stronger evidence, not weaker. |
| "Vendor is unstable, we cannot know" |
You still need M1 logs to avoid misfixes. |
| "Budget is low so scans are fine" |
Low budget means scans fail sooner. |
| "We can patch the numbers" |
Patches without replay create permanent drift. |
Red Flags - STOP
- No structured logs but attempting responsibility attribution
- Data rewrite without replay/backfill plan
- Dashboard reads from raw event tables
- Unknown fields stored without allowlist
- Idempotency key not updated when adding dimensions
1---2name: backend-mvp-guardrails3description: Use when designing or reviewing a backend MVP with tight budget, evolving schema, and reliance on third-party backends where idempotency, replay, and responsibility attribution are high-risk.4---5
6# Backend MVP Guardrails
7
8## Overview
9Minimize irreversible decisions. Every write must be idempotent, every aggregate must be replayable, and every incident must be attributable with minimal evidence.
10
11## When to Use
12- MVP backend with single-digit USD/month budget or strict capacity limits
13- Fast schema evolution or new data sources with unknown fields
14- Third-party backend dependency (e.g., InsForge) with no status page or DB metrics
15- Repeated ambiguity about whether failures are vendor or application issues
16
17When NOT to use: throwaway prototypes where data loss and misattribution are acceptable.
18
19## Core Pattern (Two Layers)
20
21### Layer 1: Principle Guardrails (platform-agnostic)
221) **Source of truth is immutable or append-only.** Avoid online recomputation on read paths.
232) **Idempotent writes.** Deterministic keys + upsert or unique constraint.
243) **Replayable aggregates.** Derived tables can be rebuilt from the source of truth.
254) **Evidence-first attribution.** No structured evidence, no blame, no destructive fix.
265) **Cost-first queries.** Pre-aggregate, cap ranges, enforce limits, avoid full scans.
276) **Schema evolution is additive.** New fields are optional and versioned; unknown fields are rejected by allowlist.
28
29### Layer 2: Platform Mapping (InsForge example)
30- **Fact table:** half-hour buckets (e.g., `vibescore_tracker_hourly`)
31- **Idempotency key:** `user_id + device_id + source + model + hour_start`
32- **Aggregates:** derived from buckets; do not read raw event tables for dashboards
33- **Retention:** keep aggregates longer; cap any event-level tables
34- **Backfill:** limited window + upsert; must be replayable
35- **Observability:** M1 structured logs (see below)
36
37## Responsibility Attribution Protocol (M1)
38**Required fields:** `request_id`, `function`, `stage`, `status`, `latency_ms`, `error_code`, `upstream_status`, `upstream_latency_ms`
39
40**Attribution rules:**
41- Missing `upstream_status` => **UNKNOWN** (do not change data semantics)
42- `upstream_status` is 5xx/timeout and function status is 5xx => likely vendor/backbone issue
43- `upstream_status` is 2xx and function status is 4xx/5xx => likely application validation/logic issue
44- `latency_ms` high and `upstream_latency_ms` low => likely application-side bottleneck
45
46**Stop rule:** no data rewrite, schema change, or semantic patch without a replay plan and rollback.
47
48## Quick Reference
49| Guardrail | Why | Minimum Implementation |
50| --- | --- | --- |
51| Idempotent writes | Prevent double-counting | Unique key + upsert |
52| Replayable aggregates | Safe fixes | Source-of-truth table + backfill job |
53| Cost caps | Fit low budget | Range limits + pre-aggregates |
54| Evidence-first | Avoid misfix | M1 structured logs |
55| Schema allowlist | Avoid data bloat | Reject unknown fields |
56
57## Implementation Example (Structured Log)
58```js
59const start = Date.now();
60const requestId = crypto.randomUUID();
61const log = (entry) =>
62 console.log(JSON.stringify({
63 request_id: requestId,
64 function: 'example-function',
65 ...entry
66 }));
67
68try {
69 const upstreamStart = Date.now();
70 const res = await fetch(upstreamUrl);
71 const upstreamLatency = Date.now() - upstreamStart;
72
73 log({
74 stage: 'upstream',
75 status: res.status,
76 upstream_status: res.status,
77 upstream_latency_ms: upstreamLatency,
78 latency_ms: Date.now() - start,
79 error_code: res.ok ? null : 'UPSTREAM_ERROR'
80 });
81} catch (err) {
82 log({
83 stage: 'exception',
84 status: 500,
85 upstream_status: null,
86 upstream_latency_ms: null,
87 latency_ms: Date.now() - start,
88 error_code: 'UPSTREAM_TIMEOUT'
89 });
90 throw err;
91}
92```
93
94## Common Mistakes
95- Online aggregation in dashboard endpoints under low budget
96- Adding new data sources without updating idempotency keys
97- Blame without `upstream_status` evidence
98- Storing full payloads "just in case" (privacy and cost risk)
99- Changing data semantics without replay/backfill plan
100
101## Rationalization Table
102| Excuse | Reality |
103| --- | --- |
104| "We are a tiny team, logs are overkill" | Small teams need stronger evidence, not weaker. |
105| "Vendor is unstable, we cannot know" | You still need M1 logs to avoid misfixes. |
106| "Budget is low so scans are fine" | Low budget means scans fail sooner. |
107| "We can patch the numbers" | Patches without replay create permanent drift. |
108
109## Red Flags - STOP
110- No structured logs but attempting responsibility attribution
111- Data rewrite without replay/backfill plan
112- Dashboard reads from raw event tables
113- Unknown fields stored without allowlist
114- Idempotency key not updated when adding dimensions