Mathguard

mathguard — Math-Heavy Optimization for AI Code

lemmaly makes you pick the right classical algorithm. mathguard kicks in when the classical algorithm is already optimal but mathematics gives a better bound — usually by accepting bounded approximation, exploiting structure, or moving to a smarter algebraic space.

The model knows these techniques. It almost never proposes them spontaneously. mathguard fixes that.

Violating the letter of these rules is violating the spirit of the skill. A Bloom filter where the caller assumed exact answers is a production incident, not an optimization.

The Iron Law

NO APPROXIMATE STRUCTURE WITHOUT WRITTEN ε/δ AND EXPLICIT CALLER ACCEPTANCE

Probabilistic data structures (Bloom, HyperLogLog, Count-Min, MinHash/LSH, t-digest), randomized projections (JL), and lossy transforms (floating FFT) all change the answer's meaning. Before proposing one:

1. Write the error parameter the caller will see (false-positive rate, relative error, distortion bound).
2. Identify the caller and state, in one sentence, that they tolerate this kind of wrong answer.
3. If you cannot identify the caller, or they need exact (auth checks, billing, dedup keys, deduplication for correctness, anything that flows into a primary key), DO NOT propose the approximate structure. Keep classical, or escalate to a sharded/streaming exact design.

This rule has saved more incidents than any other in this skill. Do not soften it.

Non-negotiable rules

1. Declare exact vs approximate up front. Before suggesting a math-level technique, state:

   • mode: exact or mode: approximate
   • If approximate: the error parameter (ε, δ, false-positive rate) and a sentence on whether the caller can tolerate it.
   • If the caller needs exact and there is no exact win, say so and stop — do not silently degrade to approximate.

2. Cite the technique by name. Never describe a probabilistic or numerical trick in vague terms. Name it: Bloom filter, HyperLogLog, Count-Min Sketch, MinHash + LSH, Johnson–Lindenstrauss projection, FFT, NTT, fast exponentiation, Karatsuba, Strassen, sweep line, kd-tree, BVH, union-find with path compression, Floyd's cycle detection, Boyer-Moore majority, reservoir sampling, Knuth shuffle, Aho-Corasick, suffix automaton, segment tree with lazy propagation, Fenwick tree, monoid scan / parallel prefix. A named technique is auditable; "a smart approximation" is not.

3. State the trade you are making. Every math-level optimization buys something at a cost. In one line:

   • Buys: space, time, wall-clock, parallelism.
   • Costs: accuracy ε=?, code complexity, dependency, non-determinism, numerical stability.
   • If the cost is invisible to the caller, write "callers see no change".

4. Justify the asymptotic win. Do not propose a math technique without a one-line bound argument:

   • "HyperLogLog: count uniques in O(log log n) bits at standard error 1.04/√m."
   • "FFT: polynomial multiplication O(n log n) vs schoolbook O(n²)."
   • "JL projection: preserves pairwise distances within (1±ε) using O(log n / ε²) dimensions."
   • "Sweep line: rectangle overlap from O(n²) pair checks to O(n log n) events." No bound, no proposal.

5. Forbid math cargo-culting. Do not introduce these techniques when:

   • n is small enough that a linear scan finishes in microseconds (n < ~10⁴ unless it is a hot path).
   • The problem is I/O-bound — the math win disappears behind network/disk.
   • Exact answers are required and no exact technique exists.
   • The team will not maintain it (write that down: "team familiarity: ?").

The pre-proposal protocol

Before suggesting a math-level technique, your message must contain — in this order:

1. The classical floor — what is the best non-mathy algorithm and its Big-O? ("Hash join is O(n+m); we're already there.")
2. Why classical is not enough — n too large, space blows up, real-time deadline, etc.
3. The math technique — named (rule 2).
4. Exact or approximate — with ε if approximate (rule 1).
5. The new bound — with one-line derivation (rule 4).
6. The trade — buys/costs (rule 3).
7. When NOT to use this — at least one disqualifier.
8. The code or pseudocode.

If any of 1–7 is missing, do not propose the technique.

Playbook — math technique → problem → win → caveat

Sketches and probabilistic structures (massive data, approximate)

Fast arithmetic / transforms (numeric and combinatorial)

Dimensionality reduction and linear algebra

Geometry (spatial queries)

Graph and algebraic tricks

Amortized and online algorithms

Canonical example — counting distinct users

Problem. Count unique users seen across a 24-hour event stream. ~2B events/day, ~50M unique users. Reported on a dashboard, ±2% is acceptable.

// "Just use a Set" — silently OOMs the box at ~50M strings
const seen = new Set<string>();
for await (const event of stream) {
  seen.add(event.userId);
}
return seen.size; // exact, but the process died at row 41M

Or worse — proposed with a HyperLogLog "for performance" but plugged into the billing pipeline, which keys off the result. Billing then sees 49.7M instead of 50.0M users and a fraction never get charged.

// Classical floor: O(unique) memory for an exact Set. At 50M strings × ~50B each, ~2.5GB.
// Why classical is not enough: dashboard box has 512MB and refreshes every minute.
// Technique: HyperLogLog (HLL).
// Mode: approximate. ε ≈ 1.04/√m. With m=2^14 registers → ~0.8% relative error.
// Trade: buys O(log log n)-bit space (~12KB); costs ±0.8% on the displayed count.
// When NOT to use: anything that flows into billing, primary keys, or per-user actions.
// Caller acceptance: confirmed — dashboard product owner accepts ±2%, written in PR.

import { createHLL } from 'hyperloglog-lite';
const hll = createHLL({ precision: 14 });
for await (const event of stream) {
  hll.add(event.userId);
}
return hll.estimate(); // 49.6M ± 0.4M; dashboard reads ~50M

The Bad version is not "no HLL" — it is "HLL without writing down ε and who tolerates it." The Good version is identical in technique but auditable: ε is in the comment, the caller is named, the disqualifier (billing) is explicit.

Output discipline

Code that uses a math-level technique must include:

• One comment naming the technique with a doc link or one-line citation.
• The exact error parameters chosen (ε, δ, bits, dimensions, etc.) and why those values.
• A measured or asymptotic justification next to the chosen parameters.
• An exact-mode fallback path, if the caller might need it.

When to escalate or redirect

• The bottleneck is I/O, not CPU/memory → go back to lemmaly rule 4; math will not help.
• You need bit-exact reproducibility → avoid floating FFT, randomized projections, and probabilistic structures.
• The result is consumed by a downstream system that assumes exact → keep classical or wrap with a validation pass.
• You need a correctness proof (not just a bound) → load invariant-guard after picking the technique.

Rationalizations to watch for

These are real verbatim thoughts captured from a controlled test where the model noticed a probabilistic structure was needed but still shipped the naive answer:

If any of these sound familiar mid-thought: stop, run the eight-step protocol, pick parameters with derivation.

Red flags — STOP

• Proposing a probabilistic structure without stating ε and δ.
• Saying "we can use FFT here" without writing the n at which FFT actually beats schoolbook.
• Using JSON.parse(JSON.stringify(...)) to deep-clone when structuredClone exists, then claiming it as an optimization.
• Recommending Strassen on a 100×100 matrix.
• Switching to approximate output without the caller having agreed to it.
• Naming a technique you cannot derive the bound for.
• Math optimization where n is small and not on a hot path.
• "Should be O(log n) on average" with no average-case argument.

All of these mean: back up, re-run the eight-step pre-proposal protocol, or fall back to lemmaly's classical playbook.

Verification checklist

Before shipping code that uses a math-level technique:

• The technique is named (no "a smart approximation").
• If approximate: ε and δ (or the equivalent error parameter) are written in code or in the PR description.
• The caller has been identified and their tolerance for that error is stated.
• A one-line bound derivation is present (asymptotic or measured).
• At least one disqualifier ("when NOT to use this") is documented.
• An exact-mode fallback exists, OR a one-line note explains why exact is impossible.
• If randomized: the seed strategy is documented (fixed for reproducibility, or stated as non-deterministic).
• Downstream consumers that assume exactness (joins on this value, billing, auth, primary keys) have been audited.

Cannot check every box? The technique is not ready to ship. Keep classical, or stop and ask.

The thesis, in one line

When classical algorithms hit their floor, mathematics still has another floor below. mathguard makes the model reach for it instead of accepting the first answer.