Off Nominal Review

Off-Nominal Review

Mission

Review a system the way a mission or operations team would: not only for the happy path, but for off-nominal conditions, unexpected environment changes, unsafe transitions, missing safeguards, and weak containment boundaries.

This is a public, software-oriented adaptation of NASA systems thinking. It is not a formal hazard analysis or safety certification method.

Source basis

This skill is shaped by the NASA Systems Engineering Handbook, Rev 2 (NASA/SP-2016-6105 Rev2). Read references/nasa-principles.md before using the skill for a deeper review.

When to use

Use this skill when the system has material workflows, lifecycle transitions, or operator-facing recovery decisions, including:

• daemons and workers
• controllers and state machines
• import, export, or migration tools
• orchestration and deployment workflows
• pipelines with partial outputs or retries
• systems that claim degraded operation or fault tolerance

Review posture

Model the target as a set of operational scenarios. For each material workflow:

1. Define the nominal path.
2. Enumerate off-nominal conditions.
3. Identify the safeguard, containment, or operator signal that should exist.
4. Trace whether that mechanism is actually implemented.
5. Record both findings and non-findings.

Off-nominal scenario inventory

Check at least these conditions where relevant:

• required dependency absent
• configuration inconsistent with environment
• operator chooses the wrong target or mode
• timeout, empty response, or malformed response
• delayed but not failed dependency
• partial write or interrupted state transition
• duplicate message, job, or rerun
• stale cache, checkpoint, or lock
• cleanup path not reached
• degraded mode entered without clear signal

Scenario worksheet

Build a worksheet before findings:

The required safeguard can be technical, procedural, or operator-visible, but it must be explicit.

Review angles

Inspect each workflow from these angles:

• transition into work
• transition out of work
• partial progress boundary
• operator decision point
• dependency instability
• containment of bad state
• recoverability after interruption
• visibility to the next operator

Guard questions

Ask questions like:

• What is the first irreversible side effect?
• What must be true before that point?
• What happens if that assumption fails one step later instead?
• What evidence would tell the operator the workflow is now off-nominal?
• Can the system contain the problem locally, or does it leak ambiguity downstream?
• What state must be preserved, rolled back, quarantined, or recomputed?

Expected safeguard vocabulary

Use concrete safeguard terms when appropriate:

• interlock
• preflight
• hold point
• explicit degrade
• quarantine
• containment boundary
• rollback point
• operator acknowledgment
• restart barrier

Finding format

Use this format:

ONR-XXX: Short title

• Severity: Low / Medium / High / Critical
• Workflow: affected workflow or transition
• Off-nominal trigger: unexpected condition
• Required safeguard: the control or signal that should exist
• Current handling: what exists today
• Evidence: code path, config path, procedure, or observed behavior
• Impact: operational or integrity consequence
• Recommended change: the narrowest change that restores the safeguard
• Tests: scenario or regression test that should prove the safeguard

Non-findings

Record explicit non-findings when the system appears to:

• reject unsafe entry conditions
• contain bad state before propagation
• surface degraded mode honestly
• preserve clear ownership of operator decisions
• avoid ambiguous partial completion

Output contract

Produce:

1. a short posture summary
2. the scenario worksheet
3. findings
4. explicit non-findings
5. a prioritized safeguard backlog
6. tests or drills that would strengthen confidence
7. limits of analysis

Limits

• Do not present this as formal FMEA, STPA, or certification evidence.
• Do not overclaim runtime behavior without concrete evidence.
• Keep the review anchored to implemented safeguards, not abstract safety language.