idea-mining-loop
Reverse-engineered from how real results were found: an idea is not chosen, it is what survives. The
loop turns "numbers look odd" into "theorem + bound", and it is designed to let most candidates die cheaply.
Worked, runnable example (public textbook fact, Gold exponents): python3 skills/idea-mining-loop/scripts/toy_mining_demo.py.
It shows an honest Gate-B failure (even n), an anomaly, a naive hypothesis killed by a control experiment,
a real-cost metric, and a matching lower bound.
When to use / not use
Use: a performance/complexity bottleneck you can compute in a few lines; 1–2 intuitive attempts already stalled;
you suspect a design constant (modulus, base, radix, ordering, round count, chunk size, hash count, folding factor)
was fixed by habit rather than proven optimal. Do not use when you already have a precise theorem target —
just prove it.
Pre-gate 0 — security-bearing object or computational substrate? (5 min)
Write one sentence: "The object I want to compress/restructure is ___; it is / is not part of the scheme's
security definition, because ___."
- Security-bearing (public key map, trapdoor/sampler output distribution, anything the adversary sees and
must look random): exclude structurally. Any exploitable structure you find there is a cryptanalytic result
(an attack), not a free optimisation. History: extra algebraic structure on public maps / trapdoors is the
usual source of breaks.
- Substrate (field arithmetic, key-switching bookkeeping, packing/transform scheduling, NTT layout, OT/OKVS
plumbing): allowed — but passing this gate is necessary, not sufficient. Step 6 is still mandatory.
If you cannot write the sentence, the target is not specific enough: go back to Step 0.
The steps (every step writes a dated note notes/<slug>_<YYYY-MM-DD>.md; never overwrite old notes)
| step |
do |
time box |
output |
| 0 Probe target |
State the bottleneck as a number computable in a few lines + baseline value + success criterion. Bad: "bootstrapping is slow". Good: "number of non-linear multiplications to evaluate map M on domain D; baseline 40". |
5 min |
probe_target note |
| 1 Cheap probes |
One ≤50-line offline script per candidate direction; answers only "is there any hope in principle?". Kill if the small-scale probe cannot approach the target, or the gain is dwarfed by overhead. Keep negative results. |
≤30 min/direction |
<direction>_probe note (also for dead ones) |
| 2 Representation tricks |
Read neighbouring work asking "what did they turn from FIXED into a FREE variable?" — one line per paper. |
1–2 h |
representation_tricks note |
| Gate A |
Name the parameter that is fixed by history/engineering default, not by proof. Cannot name one → back to 2, or the direction has no minable freedom. |
|
line in IDEAS.md |
| 3 Parametrise + sweep |
Make it a CLI parameter; sweep a range deliberately wider than intuition; cheap proxy metric first; structured output (JSON/CSV), not logs. |
≤1 h |
<param>_planner.py + results/<param>_sweep/ |
| 4 Exact synthesis |
On survivors, let a solver (SAT/SMT/ILP/exhaustive search) find the true optimum (min terms, min depth, min mults, min rounds). Per-call timeout ≤5 min. Re-verify every solver model with independent code. |
≤2 h |
table: candidate → optimum → re-verified? |
| Gate B |
Look for an outlier — "better out of proportion, not explained by the other candidates" — not the best value. Smooth monotone curve → record "no anomaly", return to Step 2. |
|
IDEAS.md verdict |
| 5 Algebraic root cause |
Test which algebraic relation the outlier satisfies (order, characteristic polynomial, divisibility, subgroup/coset, self-map of the support, fixed points). Target: one provable/refutable sentence "anomaly ⟺ property P". Keep the Sage/Wolfram code. |
≤1 h |
algebraic_root_cause note + code |
| 6 Control experiment (never skip) |
Build a candidate that shares the naive explanation's surface feature but lacks property P (and vice versa). Anomaly must vanish/appear accordingly. If not → P is wrong/incomplete, back to 5. |
≤30 min |
## Control experiment section of the Step-5 note |
| 7 Real-system validation |
Wall-clock in the real library/protocol, baseline vs candidate vs a control candidate, same flags/threads/machine, interleaved runs, ≥3 repeats (median + spread). Report component and end-to-end (Amdahl), absolute and relative. |
½ day |
results/<slug>_real/ + EVIDENCE rows |
| 8 Generalise + lower bound |
(a) Replace the specific value by a family (one exponent → all k; one prime → all primes; one ring → all rings of the class) and re-run 4–7. (b) Prove how close the result is to a lower bound (counting, information-theoretic, degree, rank). "X% faster" becomes "within Y of optimal". |
1–2 d |
THEORY.md + CLAIMS.md entries |
Mini-examples across areas (toy, for orientation)
| area |
fixed-by-habit parameter (Gate A) |
cheap probe / proxy |
what an anomaly + control would look like |
| Symmetric |
S-box exponent e in x^e (textbook uses e = 3 or inverse) |
differential uniformity per cyclotomic class |
APN outliers at e = 2^k+1; control: same Hamming weight, gcd(k,n)>1 → δ = 2^gcd (see demo) |
| FHE |
gadget base B and digit count ℓ in key switching |
noise variance × #NTTs per switch |
a base where noise drops more than the ℓ-reduction explains → check if B aligns with the RNS primes; control: same ℓ, misaligned B |
| Lattice |
block size schedule in progressive BKZ |
simulated GSA slope per unit cost |
a schedule beating the smooth trend → check dimension-for-free effect; control: same schedule without the extra dims |
| MPC/PSI |
number of hash functions / load factor in cuckoo hashing or OKVS |
failure probability vs size via simulation |
sharp threshold (phase transition) → graph-theoretic cause (2-core/peeling); control: same load factor, different hash family |
| ZK |
FRI folding factor / Reed–Solomon rate |
proof size × verifier work |
a factor where both drop together → check field/domain structure; control: same factor, different domain size |
| PQC |
rejection-sampling bound / norm check in Fiat–Shamir-with-aborts |
acceptance rate × signature size |
an outlier bound → check the distribution's tail geometry; pre-gate first: the output distribution is security-bearing, so the "optimisation" must provably preserve independence from the secret |
Prompt patterns that produced real ideas (generalised)
- Read before thinking. First make the agent summarise tried / abandoned / current best. No free association in a vacuum.
- Ignition = concrete book chapter + concrete step. "Use the structures in <textbook, chapter> to attack " — never "give me ideas".
- Human step: spot the constant. For every specific object in the agent's output ask "why this one? what if it varied?" — this is Gate A and is the least automatable step.
- Exhaust and falsify. "Not only the value that worked — every member of the family, including the ones theory says should be better — run them for real."
- Reachability audit. "Check item by item whether each theoretical saving is actually reachable."
- Theorem-ise the survivor, then ask for the lower bound and "how far from it are we?".
- Measured, not predicted. Any number without a command and a log is a prediction; say so.
- Ask for the price. "Why is the speedup so large? What got worse (communication, key size, depth, precision)?"
- Let branches die, and file them. A failed branch becomes a documented negative result or a pure-theory remark — never a drag on the main claim.
Anti-patterns (from real post-mortems)
- Trusting a hand proof over a computation: an "established" theorem later found false. Verify every algebraic claim in Sage/Lean before it enters a draft.
- Skipping Step 6: correlation (a shared surface feature) mistaken for cause.
- Sweep range too narrow: outliers often sit outside the "sensible" range.
- One real-system run: that is a smoke test, not a result.
- Comparing hand-written code with a library without matching compiler flags / build options first: a large "anomaly" that is only
-march=native.
- Reporting only the ratio: the component speedup can rise while the end-to-end gain falls, because the denominator shrank.
- A cost model that ignores fan-out/hidden terms: a solver "finds" a 30× gain that an explicit simulation erases. Validate the cost model on a small instance by direct simulation before trusting any optimum.
- Structure that exists only for one specific value (an algebraic coincidence of one constant) rarely generalises; test the generalisation early (Step 8a) before building a paper on it.
Log
Append one row per run to the project's IDEAS.md table:
| date | target | freed parameter | anomaly | root cause | control passed? | real-system result | verdict |.
1---2name: idea-mining-loop3description: Run the 8-step idea-mining loop (probe, free a fixed parameter, exact/SAT synthesis, spot an anomaly, algebraic root cause, control experiment, real-system validation, generalise + lower bound) with a pre-gate and two gates. Use when a direction has a measurable bottleneck and intuition-driven attempts have stalled.4metadata:5 source: https://skillmd.com/skills/jiayaoqijia/idea-mining-loop6---78# idea-mining-loop910Reverse-engineered from how real results were found: an idea is not *chosen*, it is what **survives**. The11loop turns "numbers look odd" into "theorem + bound", and it is designed to let most candidates die cheaply.1213Worked, runnable example (public textbook fact, Gold exponents): `python3 skills/idea-mining-loop/scripts/toy_mining_demo.py`.14It shows an honest Gate-B failure (even n), an anomaly, a naive hypothesis killed by a control experiment,15a real-cost metric, and a matching lower bound.1617## When to use / not use1819Use: a performance/complexity bottleneck you can compute in a few lines; 1–2 intuitive attempts already stalled;20you suspect a design constant (modulus, base, radix, ordering, round count, chunk size, hash count, folding factor)21was fixed by habit rather than proven optimal. Do **not** use when you already have a precise theorem target —22just prove it.2324## Pre-gate 0 — security-bearing object or computational substrate? (5 min)2526Write one sentence: *"The object I want to compress/restructure is ___; it is / is not part of the scheme's27security definition, because ___."*28- **Security-bearing** (public key map, trapdoor/sampler output distribution, anything the adversary sees and29 must look random): exclude structurally. Any exploitable structure you find there is a cryptanalytic result30 (an attack), not a free optimisation. History: extra algebraic structure on public maps / trapdoors is the31 usual source of breaks.32- **Substrate** (field arithmetic, key-switching bookkeeping, packing/transform scheduling, NTT layout, OT/OKVS33 plumbing): allowed — but passing this gate is necessary, not sufficient. Step 6 is still mandatory.34If you cannot write the sentence, the target is not specific enough: go back to Step 0.3536## The steps (every step writes a dated note `notes/<slug>_<YYYY-MM-DD>.md`; never overwrite old notes)3738| step | do | time box | output |39|---|---|---|---|40| 0 Probe target | State the bottleneck as a number computable in a few lines + baseline value + success criterion. Bad: "bootstrapping is slow". Good: "number of non-linear multiplications to evaluate map M on domain D; baseline 40". | 5 min | `probe_target` note |41| 1 Cheap probes | One ≤50-line offline script per candidate direction; answers only "is there any hope in principle?". Kill if the small-scale probe cannot approach the target, or the gain is dwarfed by overhead. Keep negative results. | ≤30 min/direction | `<direction>_probe` note (also for dead ones) |42| 2 Representation tricks | Read neighbouring work asking "what did they turn from FIXED into a FREE variable?" — one line per paper. | 1–2 h | `representation_tricks` note |43| **Gate A** | Name the parameter that is fixed by history/engineering default, not by proof. Cannot name one → back to 2, or the direction has no minable freedom. | | line in IDEAS.md |44| 3 Parametrise + sweep | Make it a CLI parameter; sweep a range deliberately **wider** than intuition; cheap proxy metric first; structured output (JSON/CSV), not logs. | ≤1 h | `<param>_planner.py` + `results/<param>_sweep/` |45| 4 Exact synthesis | On survivors, let a solver (SAT/SMT/ILP/exhaustive search) find the true optimum (min terms, min depth, min mults, min rounds). Per-call timeout ≤5 min. **Re-verify every solver model with independent code.** | ≤2 h | table: candidate → optimum → re-verified? |46| **Gate B** | Look for an *outlier* — "better out of proportion, not explained by the other candidates" — not the best value. Smooth monotone curve → record "no anomaly", return to Step 2. | | IDEAS.md verdict |47| 5 Algebraic root cause | Test which algebraic relation the outlier satisfies (order, characteristic polynomial, divisibility, subgroup/coset, self-map of the support, fixed points). Target: one provable/refutable sentence "anomaly ⟺ property P". Keep the Sage/Wolfram code. | ≤1 h | `algebraic_root_cause` note + code |48| 6 Control experiment (**never skip**) | Build a candidate that shares the naive explanation's surface feature but lacks property P (and vice versa). Anomaly must vanish/appear accordingly. If not → P is wrong/incomplete, back to 5. | ≤30 min | `## Control experiment` section of the Step-5 note |49| 7 Real-system validation | Wall-clock in the real library/protocol, baseline vs candidate vs a control candidate, same flags/threads/machine, interleaved runs, ≥3 repeats (median + spread). Report component *and* end-to-end (Amdahl), absolute *and* relative. | ½ day | `results/<slug>_real/` + EVIDENCE rows |50| 8 Generalise + lower bound | (a) Replace the specific value by a family (one exponent → all k; one prime → all primes; one ring → all rings of the class) and re-run 4–7. (b) Prove how close the result is to a lower bound (counting, information-theoretic, degree, rank). "X% faster" becomes "within Y of optimal". | 1–2 d | THEORY.md + CLAIMS.md entries |5152## Mini-examples across areas (toy, for orientation)5354| area | fixed-by-habit parameter (Gate A) | cheap probe / proxy | what an anomaly + control would look like |55|---|---|---|---|56| Symmetric | S-box exponent e in x^e (textbook uses e = 3 or inverse) | differential uniformity per cyclotomic class | APN outliers at e = 2^k+1; control: same Hamming weight, gcd(k,n)>1 → δ = 2^gcd (see demo) |57| FHE | gadget base B and digit count ℓ in key switching | noise variance × #NTTs per switch | a base where noise drops more than the ℓ-reduction explains → check if B aligns with the RNS primes; control: same ℓ, misaligned B |58| Lattice | block size schedule in progressive BKZ | simulated GSA slope per unit cost | a schedule beating the smooth trend → check dimension-for-free effect; control: same schedule without the extra dims |59| MPC/PSI | number of hash functions / load factor in cuckoo hashing or OKVS | failure probability vs size via simulation | sharp threshold (phase transition) → graph-theoretic cause (2-core/peeling); control: same load factor, different hash family |60| ZK | FRI folding factor / Reed–Solomon rate | proof size × verifier work | a factor where both drop together → check field/domain structure; control: same factor, different domain size |61| PQC | rejection-sampling bound / norm check in Fiat–Shamir-with-aborts | acceptance rate × signature size | an outlier bound → check the distribution's tail geometry; **pre-gate first**: the output distribution is security-bearing, so the "optimisation" must provably preserve independence from the secret |6263## Prompt patterns that produced real ideas (generalised)64651. **Read before thinking.** First make the agent summarise *tried / abandoned / current best*. No free association in a vacuum.662. **Ignition = concrete book chapter + concrete step.** "Use the structures in <textbook, chapter> to attack <specific algorithm step>" — never "give me ideas".673. **Human step: spot the constant.** For every specific object in the agent's output ask "why this one? what if it varied?" — this is Gate A and is the least automatable step.684. **Exhaust and falsify.** "Not only the value that worked — every member of the family, including the ones theory says should be better — run them for real."695. **Reachability audit.** "Check item by item whether each theoretical saving is actually reachable."706. **Theorem-ise the survivor**, then **ask for the lower bound** and "how far from it are we?".717. **Measured, not predicted.** Any number without a command and a log is a prediction; say so.728. **Ask for the price.** "Why is the speedup so large? What got worse (communication, key size, depth, precision)?"739. **Let branches die, and file them.** A failed branch becomes a documented negative result or a pure-theory remark — never a drag on the main claim.7475## Anti-patterns (from real post-mortems)7677- Trusting a hand proof over a computation: an "established" theorem later found false. Verify every algebraic claim in Sage/Lean before it enters a draft.78- Skipping Step 6: correlation (a shared surface feature) mistaken for cause.79- Sweep range too narrow: outliers often sit outside the "sensible" range.80- One real-system run: that is a smoke test, not a result.81- Comparing hand-written code with a library without matching compiler flags / build options first: a large "anomaly" that is only `-march=native`.82- Reporting only the ratio: the component speedup can rise while the end-to-end gain falls, because the denominator shrank.83- A cost model that ignores fan-out/hidden terms: a solver "finds" a 30× gain that an explicit simulation erases. Validate the cost model on a small instance by direct simulation before trusting any optimum.84- Structure that exists only for one specific value (an algebraic coincidence of one constant) rarely generalises; test the generalisation early (Step 8a) before building a paper on it.8586## Log8788Append one row per run to the project's `IDEAS.md` table:89`| date | target | freed parameter | anomaly | root cause | control passed? | real-system result | verdict |`.