tomevault-io Operability Guideline
Tech Stack
- Workflows: Upstash Workflows + QStash
- Cache: Upstash Redis
- Platform: Vercel
Non-Negotiables
- Dead-letter handling must exist and be operable (visible, replayable)
- Side-effects (email, billing, ledger) must be idempotent or safely re-entrant
- Drift alerts must have remediation playbooks
Context
Operability is about running the system in production — not just building it. Systems fail. Jobs get stuck. State drifts. The question is: when something goes wrong, can an operator fix it without deploying code?
Consider the operator experience during an incident. What tools do they have? What runbooks exist? Can they safely retry failed jobs? Can they detect and fix drift?
Driving Questions
- What happens when a job fails permanently?
- How would an operator know something is stuck?
- Can failed workflows be safely replayed without duplicating side-effects?
- What drift can occur between systems, and how would we detect it?
- What's the rollback plan if a deploy breaks something critical?
- What runbooks exist, and what runbooks should exist but don't?
Converted and distributed by TomeVault — claim your Tome and manage your conversions.