Threat Modeling

Threat Modeling

Overview

Threat modeling — structured enumeration of "what could go wrong" for a system, BEFORE it ships. The goal is not perfection but calibration: forcing the design team to name the attacker, the assets, the trust boundary, and the controls so that defenses are commensurate with what's at stake.

This skill ships three methodologies; pick by stakes and data class:

Companion skills:

• llm-security — agentic-system threats (LLM01-LLM10) — invoke INSTEAD of generic STRIDE for LLM-tool-use systems
• python-auth-security — implementation patterns for auth-related mitigations
• secret-scanning / sast-tooling — runtime defenses for secret-class threats
• forge Step 1 security branch — invoke this skill from the security branch when high-stakes design is detected

1. STRIDE (the workhorse)

STRIDE categorises threats by what they do to a system. Six categories, easy to remember:

STRIDE per data-flow element

A standard threat-modeling exercise runs STRIDE per element of the data-flow diagram (DFD):

STRIDE template (Markdown)

## Component: <name>
**Stakes:** {low | medium | high | critical}
**Trust boundary:** <describe inside vs outside>
**Attacker model:** <who, what capabilities, what motivation>

| ID | Element | STRIDE | Threat | Mitigation | Status |
|---|---|---|---|---|---|
| T-01 | API endpoint POST /transfer | S | Forged caller identity | OAuth2 PKCE + tenant claim validation | Implemented |
| T-02 | API endpoint POST /transfer | T | Replay of legitimate request | Nonce + idempotency key | Implemented |
| T-03 | API endpoint POST /transfer | R | User denies action | Append-only audit log + signed receipt | Implemented |
| T-04 | API endpoint POST /transfer | I | Response leaks balance to unauthorised | RBAC check on account.owner_id | Implemented |
| T-05 | API endpoint POST /transfer | D | Rapid-fire transfers exhaust DB connections | Rate limit (10/min/user), circuit breaker | Implemented |
| T-06 | API endpoint POST /transfer | E | Caller escalates to admin via JWT manipulation | Validate `aud`, `iss`, `tid`, `azp` claims; signing-key rotation | Implemented |

**Accepted risks (with sign-off):**
- T-07: TOCTOU race between balance check and debit. Accepted: locked at row-level via SELECT FOR UPDATE; very-fast-path may still race under extreme contention. Mitigated by reconciliation job. Signed-off: <engineer>, <date>.

2. LINDDUN (privacy)

LINDDUN extends STRIDE for privacy-sensitive systems. Seven categories:

Use LINDDUN when:

• Component handles PII directly (names, addresses, IPs, biometrics, financials, health)
• Regulatory regime requires it (GDPR, CCPA, HIPAA, PIPEDA, LGPD)
• The system claims "anonymous" or "private" in marketing
• High-stakes social/political/employment data

LINDDUN template addition (extends STRIDE table)

| ID | Element | LINDDUN | Threat | Mitigation | Status |
|---|---|---|---|---|---|
| P-01 | Analytics event store | L (Linkability) | Two pseudonymous events can be linked via IP+device fingerprint | IP truncation, no device-fingerprint storage | Implemented |
| P-02 | User profile API | I (Identifiability) | Quasi-identifier set (DoB + ZIP + sex) re-identifies the user | k-anonymity k≥5 on returned aggregations | Implemented |
| P-03 | Login event log | D (Detectability) | Existence of a record reveals user activity on a sensitive date | Login events purged after 90 days | Implemented |
| P-04 | Consent management | U (Unawareness) | User isn't aware of secondary data use | Granular consent screen + change-of-purpose notification | Implemented |
| P-05 | Marketing pixel | N (Non-compliance) | GDPR Art. 6 lawful basis not documented | Cookie consent banner + lawful-basis matrix in data inventory | Implemented |

3. Kill chain / MITRE ATT&CK (high-stakes)

For production-critical / customer-facing-trust components, map adversary behaviour to kill-chain phases. Use MITRE ATT&CK technique IDs for the actionable detail.

Kill chain phases

Kill-chain template addition

## Adversary scenarios (high-stakes)

### Scenario A: External attacker, opportunistic
**Profile:** Skilled but not-targeted; scanning for vulns.

| Phase | Likely technique | Our detection | Our prevention |
|---|---|---|---|
| Reconnaissance | T1595 — active scanning of public endpoints | WAF logs + rate-anomaly alert | Public-facing endpoint minimisation |
| Initial access | T1190 — exploit public-facing app (e.g. dep CVE) | dep-currency-check + WAF | Patched deps; minimal-privilege service |
| Execution | T1059 — command-and-scripting interpreter | EDR + audit log | Read-only FS; no shell access |
| Privilege escalation | T1078.004 — cloud accounts | CloudTrail anomaly | Least-privilege IAM |
| Credential access | T1552.001 — credentials in files | secret-scanning + EDR | No secrets in code (gitleaks) |
| Exfiltration | T1567.002 — exfil to cloud storage | egress allowlist + DLP | Egress allowlist + tenant-DLP |

### Scenario B: Insider (employee, contractor, sub-processor)
**Profile:** Authorised baseline access; abusing privilege.

| Phase | Likely technique | Our detection | Our prevention |
|---|---|---|---|
| Discovery | T1087 — account discovery | Audit log review | RBAC minimisation |
| Collection | T1213 — data from info repository | DLP + access logging | Least-privilege; data classification |
| Exfiltration | T1567 — exfil via web service | Egress monitoring | Egress allowlist; sensitive-data flag |

4. Mitigation classes — what defenses look like

When a threat is identified, the design must respond with one of:

Every threat in the table MUST map to at least one mitigation class. An entry of just "we'll be careful" is a defect — replace with concrete control or accepted-risk sign-off.

5. When to invoke (decision tree)

Does the component process untrusted input OR hold secrets/tokens
OR cross a trust boundary OR have elevated privileges?
│
├── NO → No formal threat model needed. Apply baseline security
│        practices (TLS, auth, audit log). Document a one-paragraph
│        "low-stakes; STRIDE deferred" note in the design.
│
└── YES → Is it an LLM-tool-use / agentic system?
        │
        ├── YES → Use `llm-security` (OWASP LLM Top 10 + Dual LLM
        │        architecture) INSTEAD of generic STRIDE. The skill
        │        already structures the threat space for that class.
        │
        └── NO  → Does it handle PII / health / financial / regulated data?
                │
                ├── YES → STRIDE + LINDDUN (§1 + §2)
                │
                └── NO → Is it high-stakes (production-critical,
                         customer-trust, irreversible action)?
                        │
                        ├── YES → STRIDE + Kill-chain (§1 + §3)
                        │
                        └── NO → STRIDE only (§1)

6. Output template (decision-grade)

# Threat Model: <component-name>

**Date:** 2026-MM-DD
**Author(s):** <engineer>, <reviewer>
**Component:** <path-or-URL>
**Methodology:** STRIDE [+ LINDDUN] [+ Kill-chain]
**Stakes tier:** low | medium | high | critical
**Estimated time-to-mitigate:** <hours-or-WPs>

## 1. Trust boundary
<one paragraph describing inside vs outside, with diagram if non-trivial>

## 2. Attacker model
**Who:** <profile — external, insider, adversary state-level, opportunistic>
**Capabilities:** <what they can already do — typed inputs, network access, internal credentials>
**Motivation:** <financial gain, data theft, sabotage, espionage, harassment>

## 3. Assets
| Asset | Sensitivity | Where stored | Access path |
|---|---|---|---|

## 4. Threats — STRIDE
<the STRIDE table from §1>

## 5. Threats — LINDDUN (if applicable)
<the LINDDUN table from §2>

## 6. Adversary scenarios (high-stakes only)
<kill-chain scenarios from §3>

## 7. Design decisions driven by this model
- Decision 1: <what changed in the design because of this exercise>
- Decision 2: ...
- Decision 3: ...
(MUST have at least 3 — see HARD-RULE 1)

## 8. Accepted risks (with sign-off)
- Risk 1: <description> — signed by <engineer>, <date>
- Risk 2: ...

## 9. Open items
- Item 1: needs further investigation in <area>
- Item 2: ...

## 10. Reassess trigger
This model is valid as of <date>. Reassess if:
- New trust boundary added
- Asset sensitivity changes
- Major dependency upgrade
- Production incident
- Annual review

7. Worked example — a sketch

# Threat Model: cross-project-mail v1
**Stakes tier:** medium (single-host trust model; agents-not-humans)
**Methodology:** STRIDE

## 1. Trust boundary
Single Linux host, single user. All projects share the user's filesystem trust.
Inside: any agent invocable as the same UID. Outside: other UIDs, the network,
agents in different containers.

## 2. Attacker model
- Compromised agent in a sibling project (prompt-injected via a tool result it
  read), trying to use cpmail to influence agents in a more-privileged project.
- Mistyped sender claim (`source_type: human` when content is actually scraped
  web text).

## 3. Threats — STRIDE
| ID | Element | STRIDE | Threat | Mitigation | Status |
|---|---|---|---|---|---|
| T-01 | cpmail read | S | Sender claims `source_type: human` when actually scraped | Recipient agent treats body via `<user_data>` wrap regardless of source_type | Implemented |
| T-02 | cpmail file storage | T | Stale mail tampered post-send | Append-only; no edit API | Implemented |
| T-03 | cpmail audit | R | Sender denies sending | (Single-user model: acceptable; v1 scope) | Accepted — see history |
| T-04 | cpmail read body | I | Mail body contains scraped tokens | Recipient must NEVER eval body content (HARD-RULE) | Implemented |
| T-05 | cpmail spam | D | Sibling spams inbox | Per-project quota (deferred to v2) | Accepted — single-user low-stakes |
| T-06 | cpmail escalation | E | Sender's content steers recipient to use privileged tools | Recipient must read via `cpmail read` (delimiter wrap); no direct `cat` | Implemented |

## 7. Design decisions driven
1. `<user_data>` delimiter wrap on every body read (mitigates T-01, T-04, T-06)
2. `source_type` enum with required honest declaration (mitigates T-01)
3. v2 deferred queue authentication for cross-host (acknowledges T-03 + T-05)

This is roughly the model that produced the actual cross-project-mail skill. Notice every threat resolved to either a concrete mitigation or an explicit accepted-risk.

8. Anti-patterns

9. Selection Cheatsheet

• Building an LLM agent that has tool access → llm-security skill (Dual LLM architecture). Not this one.
• Building a new REST API endpoint → STRIDE (§1). 30-60 min exercise.
• Building a feature touching user PII → STRIDE + LINDDUN (§1 + §2). 60-90 min.
• Building production-critical / customer-trust component → STRIDE + Kill-chain (§1 + §3). 90-150 min.
• Building both: PII-handling production-critical → STRIDE + LINDDUN + Kill-chain. ~2 hours.
• Refactoring an existing component without expanding the trust boundary → No new model. Verify the existing one still applies.
• Quarterly security review → Re-walk existing TMs against new ATT&CK techniques and new CVEs surfaced by dep-currency-check.

10. Gotchas

11. Update triggers (alf scans these)

• New OWASP Top 10 / OWASP LLM Top 10 / OWASP ASVS edition (re-map categories)
• New MITRE ATT&CK techniques added (or technique renamed)
• New LINDDUN guidance from the methodology working group
• New regulatory regime (e.g. EU AI Act, US AI EO updates)
• Major incident in the wild attacking a similar component class
• Annual review on 2027-05-25

12. See Also