Gild Triage

Gild Triage: Do the Founding Assumptions Hold?

Every project rests on a set of assumptions ("this problem exists", "incumbents can't solve it", "this way is better"). This triage answers one question: do those assumptions survive evidence? When they fail, code quality and engineering discipline change nothing; code is opened only as evidence for or against an assumption.

Principles

• Gilding is zero signal. With code generators everywhere, tidy naming, tasteful comments, and a beautifully typeset README no longer reflect effort or competence — polish is free. Signal lives only in the gap between what is claimed and what evidence supports.
• Read-only. Diagnose, don't prescribe: no fixes, no modifications to the project under evaluation. Write serves only to emit the report.
• Targeted evidence, fixed budget. Open code only to test a specific assumption: at most 3 code sites per assumption, 15 sites total. What the budget cannot settle gets [ABSENT] and goes on the escalation list.
• Absence of evidence ≠ counter-evidence. Looked where one should and found nothing (project too young, community too small, data unreachable): record [ABSENT] with the reason, weighted by project maturity. Only an actual observation that contradicts the assumption counts as counter-evidence.
• Traceable. Every conclusion points back to specific evidence: a doc quote, a code location, a search result.
• Three statement types, tagged explicitly in the report:
  • [OBSERVED] — a fact seen directly (a file exists, a search returned zero hits).
  • [INFERRED] — a logical derivation from observations.
  • [JUDGMENT] — a subjective estimate. Every judgment ships its full reasoning chain: grounds + uncertainty + conclusion. A bare yes/no is not a judgment.
  • [MECHANICAL] (used in Step 2) tags a check type, not a fourth statement kind: its result is an [OBSERVED] fact.

Orient to the decision

Confirm the user's decision scenario first; it sets the scope:

• Adopt / select: full run.
• Inherit / forced maintenance: existence is already a fact; test only the assumptions that still steer future maintenance.
• Author self-audit: full strictness; a self-declared experiment earns no leniency.

Step 1: Extract the assumption tree

Assemble the project's reasons to exist into a dependency tree; mark the nodes whose failure collapses everything.

• The null hypothesis (holds by default for every project, written down or not): the strongest incumbent plus a little glue code solves the problem this project targets. The project's entire case rests on overturning it; the burden of proof sits with the project. This is one of the two points that search outside the repo (the other: Step 3's behavior evidence):
  • Find the 3–5 nearest incumbents (mind the model's knowledge cutoff; have a human re-check the comparison set for high-stakes selection).
  • Does the documentation compare against incumbents? A missing comparison chapter means the author either doesn't know the incumbents or won't face them — record it either way.
  • Is the comparison fair? Benchmarking against "hand-rolling everything" while dodging the strongest incumbent is a strawman comparison.
• Documented assumptions: extract every X from each "because X, therefore Y" in the README/design docs, place it in the tree, and tag its evidence state: verified / narrative-only / [ABSENT] / contradicted.
• Trend as argument: "macro trend T is coming, so a rewrite is needed" argues nothing. Press: which concrete user problem does the new approach solve that the strongest incumbent cannot? No answer anywhere in the docs = zero evidence for that node.
• The user assumption: the hardest test of "this pain is widespread" is people other than the author — strangers in issues/discussions, downstream dependents, deployment evidence. All activity from the author alone: a project with enough age and exposure yet no external users → counter-evidence; a young project → [ABSENT].

Done when: the dependency tree (as text) exists and every node carries an evidence state.

Step 2: Test each assumption

Return to the code — documents alone cannot test any assumption. Six failure modes follow, each tagged [MECHANICAL] (a directly checkable fact) or [JUDGMENT] (delivered per the judgment contract in Principles).

Testing is two-sided: for each critical assumption, first write its steelman — the strongest real evidence in its favor (a genuine differentiating implementation in the code, a concrete scenario incumbents truly cannot cover) — then press with the failure modes. A "fails" verdict requires failure evidence that clearly outweighs the steelman, not any single blemish.

• Plugin test [JUDGMENT] — the main instrument against the null hypothesis. Could every substantive difference between this project and the strongest incumbent be delivered as an extension, plugin, or a little glue code on that incumbent?
  • This is a feasibility estimate, not a measurement. Grounds must cover: which extension points the incumbent offers, the technical nature of the differences, and for whom the verdict holds — the sliver incumbents cannot reach may be the whole of some user's use case.
  • A well-grounded "cheap to replicate" = the null hypothesis stands and the case for existence fails.
• Misplaced comparison [JUDGMENT]: for each claimed advantage, whom is it really over? Common shape: an advantage over class-A solutions marketed as an advantage over class B, when class B never had the claimed problem. State whom the advantage actually beats and whom the marketing implies.
• Guarantees that never bore load [MECHANICAL+JUDGMENT]: invariants, tests, and gates all green — because the scenario they guard does not exist yet.
  • Mechanical: trace the data flow to the code path where the assumption actually fires; an unwired path is a checkable fact. So is a stub or TODO sitting on that load-bearing path — record them while tracing (stubs at the periphery are normal construction; ignore them).
  • Judgment: estimate how the guarantees degrade when real load arrives, with grounds.
• Claim–implementation gap [MECHANICAL]: search the code for each headline claim's implementation and tests. Zero hits is a mechanical fact — the claim lives only in the docs. Same ledger: claimed invariants with no validation, type constraint, or test (claimed ≠ enforced); differentiation described entirely in adjectives with no measurable noun; "general-purpose" claims backed only by a minimal demo; badges and version numbers ahead of substance.
• Failed its own gate [MECHANICAL]: the project set a gate for itself (a benchmark to pass, acceptance criteria, "proceed only after X") and proceeded without passing. Judge it by its own standard; the result quotes itself.
• Internal contradiction [JUDGMENT, quotes required]: one part of the design recreates the very problem another part explicitly set out to solve; or two selling points cannot hold in the same scenario. Quote both sides — original text or code locations.

Done when: every critical node records a steelman plus at least one failure-mode result, each verdict backed by evidence (doc quote + code location), and every [JUDGMENT] ships its grounds.

Step 3: Press with real user behavior

This entire step is [JUDGMENT]; name the observation sources.

For each critical assumption that survived Step 2, press once more with the target users' existing behavior: how do they actually use the incumbents? One recurring pattern: users already pay a visible, ongoing cost to keep some property — and the project's assumption requires giving that property up. Existing paying (or effort-paying) behavior is the hardest demand evidence; an assumption that contradicts it is countered. Economics — cost, latency — count as behavior too.

Every behavioral claim carries a checkable source (issue/discussion link, public data). When behavior data is unreachable (niche domain, closed ecosystem), record [ABSENT] for this step and draw no conclusion.

Done when: every surviving critical assumption has been pressed once, and counter-evidence or [ABSENT] is written back to its node.

Step 4: Residual value and verdict

Strip away every failed assumption: what remains? That residue is often the product the project should actually be. If the author explicitly rejects that positioning in the docs, record it as the top risk.

Before assigning a tier, fill the cost ledger — the grounds the Valuable but costly tier draws on (without it that tier is an ungrounded judgment):

• dependency surface, abstraction layers [OBSERVED]
• bus factor, maintenance cadence [OBSERVED]
• adopter lock-in and migration cost [JUDGMENT]
• opportunity cost: the same effort spent on the strongest incumbent (a PR, a plugin) [JUDGMENT]

Verdict tiers (the tier itself is a [JUDGMENT]; attach one sentence of grounds):

The triage targets the gap between claims and evidence, not the motive to build. Building for joy is fine; packaging an experiment as a solution and selling it is the problem.

Report

1. Assumption dependency tree: each node with its evidence state, test result, and evidence sources.
2. Tier + one-sentence grounds + a direct recommendation for the user's current decision + a description of the residual shape.
3. Falsifiable signals: for each doubtful assumption, one future observable event (how a class of issue gets handled, a benchmark result, whether a promised gate is honored) that will settle in one look whether the assumption is discipline or slogan.
4. Escalation list: which doubts deserve a deeper audit, and which key evidence is currently [ABSENT].

Length: apart from the tree, keep the body within one page.