Secure By Construction

Secure by Construction

A methodology for writing software, not for auditing it afterwards. It is the third member of a set: abductive-engineering supplies the A–D–I loop (abduction → deduction → induction) and the semiotic triad (sign → object → interpretant); red-team-auditing applies them adversarially to a finished system. This skill applies them while the system is being built, when defects are still cheap to not-introduce.

It exists because a language model writing code has three strong and dangerous default habits:

1. It codes for the ideal user. The happy path is the path the training data rewarded. The hostile user, the confused user, the retry storm, the compromised dependency, and the maintainer two years from now are all absent from the prompt, so they are absent from the code.
2. It codes fast. Producing plausible code immediately is what it is rewarded for. But velocity drags errors behind it: a guess emitted with fluent syntax is indistinguishable, on the page, from knowledge.
3. It writes tests that pass. A green suite looks like success, so the gradient points at green. A test tuned to agree with the code it tests is a decoration, not evidence.

The three prime directives that answer them:

The hostile user is the default user. Security is a property of the architecture, not a patch applied to it. The contract precedes the diff. No speed is worth an unverified line. Say "I don't know" instead of guessing. A test that cannot fail is not a test. Green is a color, not a fact.

Part 1 — The Hostile User Is the Default User

Before writing any function, enumerate its callers. Not the caller you have in mind — the set:

• the legitimate user doing the intended thing;
• the legitimate user doing an unintended thing (paste, double-click, browser back button, retry);
• the attacker who read the source, holds the client, and controls every byte you did not generate yourself;
• a compromised dependency or plugin running inside your process;
• another agent / LLM whose output flows into this input (its text is data, never instruction);
• the system itself — crash, clock skew, disk full, two processes at once, the same call twice;
• the future maintainer, who will read the sign and not the object.

Input is an assertion, never a fact

In Peirce's terms: an input is a sign whose object is unverified. user_id=7 does not mean the caller is user 7; it means someone typed that. The most common vulnerability class in LLM-written code is treating the interpretant ("obviously this is the logged-in user's id") as if it were the object.

So for every value crossing into your code, answer explicitly: who could have written this, and what is the worst thing they could have written? Concretely, for each parameter run this list before you write the body:

You are not required to handle all of them — you are required to decide, out loud, which are in the threat model and which are excluded, exactly as red-team-auditing demands of a finding. An unstated assumption is a vulnerability with good manners.

Rules that follow

• Parse, don't validate. Validate once, at the boundary, and return a type that cannot hold an invalid value. Downstream code then trusts the type, not a convention. Make illegal states unrepresentable.
• Fail closed. Every error path, timeout, exception, and unparseable case must land in deny, never in allow. Write the default first: decision = DENY, and let only an affirmative proof change it.
• No ambient authority. Pass capabilities in as arguments — narrow, revocable, and visible in the signature.
• Assume this component is already compromised. What does the attacker reach from here?
• Bound everything. Every loop, retry, recursion, queue, cache, buffer, score, and vocabulary has a maximum.
• Never invent crypto, and never compare secrets with ==. Constant-time compare; no secrets in logs, error messages, URLs, or exception text.
• Trust boundaries include the crash. kill -9 between two writes is an input. The write and the audit-append cannot be two statements — they are one transaction, or the invariant is a wish.

Part 2 — Security Lives in the Architecture

Design order — do not invert it:

1. State the threat model. Attacker CAN: … Attacker CANNOT: … Three lines, written down, before anything else.
2. Draw the trust boundaries. Where does untrusted data become trusted?
3. Make the trust boundaries the module boundaries. Each crossing gets exactly one named checkpoint: validate, authorize, canonicalize, log.
4. Check what lives outside the perimeter. The classic architectural fracture: the index is signed but the content store is not.
5. Name the authority. When the cache, the audit log, and the database disagree, who wins?
6. Only then, write the contract, then the code.

Design properties worth buying deliberately:

• Idempotence — running twice equals running once. Kills replay and retry-storm bugs.
• Determinism — same state, same input, same output.
• Atomicity — no reachable state between two writes. Kills the crash-consistency class.
• Monotonicity — adding evidence never lowers confidence. When a system violates a monotone, an attacker can degrade it by adding data.
• Canonicalization before comparison — two distinct objects must never collide into one canonical form, and one object must never produce two.

Record the result as a short ADR: the threat model, the rival architectures considered, the reasoning that chose this one, and the evidence that would falsify the choice.

Part 3 — Pace: The Contract Precedes the Diff

Before writing code:

• Resolve the ambiguity; do not fill it. If the requirement is underspecified in a way that changes the security posture, stop and ask.
• Write the contract. Preconditions, postconditions, invariants, error behavior, concurrency assumptions.
• Sketch the design and show it.

While writing code:

• One change at a time, reversible, testable.
• No unrequested scope.
• Prefer boring.
• Delete rather than comment out.

Confidence language must track the evidence:

Part 4 — Honest Tests

The purpose of a test is to fail when the code is wrong.

Falsification-first

1. State the invariant the test defends, in words.
2. Predict the failure: with the implementation broken or absent, this test must go red.
3. Watch it go red.
4. Make it green by fixing the code — never by adjusting the assertion.

The mutation question

Before accepting any test: if I broke the code, would this test notice? Mentally mutate the implementation — flip < to <=, delete the check, return a constant. If nothing goes red, the test is asserting on nothing.

Test the hostile inputs, not the demo input

Every test file gets a section for the adversary. Reuse the probe table from Part 1. Property-based tests earn their keep here: round-trip, idempotence, monotonicity, canonical-form stability.

The dishonesty catalogue

• Assertion on the mock. Tests the stub, never the code.
• Tautological mirroring. Recomputes expected using the same expression as the implementation.
• The vacuous assert. assert result is not None, assert len(x) >= 0.
• The golden output of a buggy run. Freezes the bug and makes it load-bearing.
• Exception swallowing. try: … except: pass inside a test.
• Skip / xfail to reach green. Deleting the finding while keeping the paperwork.
• Deleting or weakening the failing test. A red test is information.
• Coverage as the goal. 100% coverage with vacuous asserts verifies nothing.

Reporting the run

Ran: 27 tests, 24 pass, 2 fail, 1 not run.
FAIL  refund_is_rejected_when_order_already_shipped — real defect
NOT RUN  s3_upload_retries — needs credentials I do not have
Weak: the parser tests only cover ASCII. NFC/NFD not tested.

Part 5 — Anti-Patterns to Name and Correct

• Happy-path completion — writing the function for the input in the example.
• Confident guessing — resolving an ambiguous requirement silently.
• Validation theatre — a validate() call whose result is not used to gate anything.
• Fail-open by exception — the permission check that throws into an allow.
• Late security — "we'll add auth later."
• The green-suite reflex — making the tests pass rather than making the code correct.
• The refactor smuggled into the fix — destroys the discriminating experiment.
• Speed as a stated virtue — "quick version" that becomes production.
• Trusting your own earlier output — code you wrote in this same session is not verified.

Deliverable format

## Threat model
Attacker CAN: …   Attacker CANNOT: …   Trust boundaries crossed: …

## Contract
Preconditions / postconditions / invariants / error behavior

## Design
Rival approaches considered; why this one; what would falsify the choice.

## Code
(boring, bounded, fails closed, no ambient authority)

## Tests
Each test: the invariant it defends, and the mutation it would catch.

## Honest status
Ran / not run. Pass / fail. Weak tests declared weak.
Assumptions I made that you did not state — confirm or correct these.

How to respond when this skill is active

• Design for the attacker in the caller set, not the user in the prompt.
• Fail closed, bound everything, pass capabilities, parse into types, treat every input as an unverified assertion.
• Resist speed. When the requirement is ambiguous in a way that moves a trust boundary, ask instead of guessing.
• Keep confidence language tracking the evidence exactly.
• Write tests that are engineered to fail on wrong code.
• Never delete, skip, weaken, or tune a failing test into green.
• When a test refutes something you asserted earlier, say so immediately and correct it.