COLT Reproducibility
At COLT, reproducibility means a competent reader can re-derive every claim from the
PDF alone. There is no reproducibility checklist form in the COLT 2026 CFP (checked
2026-07-08) — the venue enforces the property the hard way, through referees who
attempt the re-derivation during review. This skill is the pre-submission audit that
makes that attempt succeed.
The re-derivability standard
A theorem is reproducible when all of the following hold:
- Its statement is formally complete: every symbol quantified, every assumption named
in the statement itself or cited by an explicit assumption label, the probability
space and adversary model unambiguous.
- Its proof exists in full inside the submitted PDF — COLT's unlimited appendix
removes every excuse for "omitted for lack of space."
- Each proof step is locally checkable: an expert reading line k needs only lines
1..k-1, cited external results, and standard background — never your unpublished
intuition.
- Constants and parameter regimes survive the chain: if Theorem 1 needs
n ≥ C·d·log(1/δ), the reader can trace what C is or where it was declared absolute.
Audit table: where re-derivation fails
| Failure mode |
Typical symptom in the PDF |
Repair |
| Hypothesis smuggling |
Proof uses independence never assumed |
Add the assumption to the statement, or redo the step |
| External-result misuse |
"By [X], ..." where [X] needs bounded support you don't have |
Check [X]'s hypotheses; find the right variant or prove a lemma |
| Constant drift |
C doubles silently between two displays |
Number constants (C_1, C_2, ...) and track them in a ledger |
| Case leakage |
"The case d=1 is analogous" when it is not |
Write the case or prove the reduction |
| Notation overload |
Same symbol for a filtration and a function class |
One notation table, enforced globally |
| Silent regime switch |
Bound proved for T large, quoted for all T |
State the threshold explicitly in the theorem |
Assumption bookkeeping pattern
Give assumptions their own numbered environment and cite them by label everywhere:
\newtheorem{assumption}{Assumption}
\begin{assumption}[Bounded losses]\label{ass:bounded}
For all $t$, the loss $\ell_t$ maps to $[0,1]$.
\end{assumption}
\begin{theorem}\label{thm:regret}
Under Assumptions~\ref{ass:bounded} and~\ref{ass:oblivious}, the algorithm's
regret satisfies $R_T \le 4\sqrt{T \log K}$ for all $T \ge 1$.
\end{theorem}
The payoff is auditable dependency: a reviewer can grep which theorems rely on
obliviousness, and your rebuttal can answer "is Assumption 2 needed for Theorem 3?"
with a pointer instead of an essay.
Numerical illustrations, when present
Some COLT papers plot a simulated regret curve or a phase transition to illustrate a
bound. The theory community's floor for those figures:
- Fixed seeds, stated replication counts, and error bars whose meaning the caption
defines; a noisy single run "consistent with the theory" persuades no one here.
- The simulated regime must be the theorem's regime — matching horizon, dimension, and
noise assumptions — or the mismatch must be acknowledged as exploratory.
- The generating script should be one dependency-light file, releasable after
acceptance; during review, describe the procedure precisely in the appendix since no
upload channel exists.
- Never let an illustration silently extend the claim ("the bound appears to hold for
heavy tails too") without labeling it as conjecture.
Vignette: the spine of a lower-bound paper
Consider a submission whose main result is a $\Omega(\sqrt{TK})$ lower bound via a
new instance family. Its re-derivability spine, in the order a referee will attack it:
- the instance family's construction, with every distribution parameter explicit and
the randomization protocol (oblivious vs. adaptive) named;
- the information-theoretic step (say, a KL-divergence calculation) with the exact
divergence bound displayed, not cited as "standard";
- the reduction from learner performance to the divergence quantity, where hypothesis
smuggling most often hides (does the argument secretly assume deterministic
learners? say so or generalize);
- the final optimization over instance parameters, with the maximizing choice written
out — "choosing ε appropriately" is where constants go to die.
A referee who can walk this spine without leaving the PDF marks correctness resolved;
each externalized step converts into a review question, and three review questions
into a reject.
Pre-submission re-derivation drill
- Print the numbered-statement list (all definitions, assumptions, lemmas, theorems).
- For each, a non-author coauthor answers: can I state precisely what this claims,
including quantifiers, without reading the proof? Rewrite until yes.
- Verify proofs in dependency order, marking each line verified/unverified; the
colt-artifact-evaluation skill's ledger format works here.
- Re-check every external citation against the cited source's actual hypotheses —
allocate real time; this is where careful papers die.
- Reconcile body sketches against appendix proofs: a sketch that describes an older
proof strategy than the appendix executes reads as a gap to a referee.
Cycle-volatility warnings
- If a future COLT cycle adds any checklist, code policy, or supplementary channel,
the current CFP announces it; the 2026 cycle had none (待核实 in later cycles).
- The 12-page body and single-PDF rules that shape where proofs live are the 2026
formulation; re-read the live CFP before restructuring a paper around them.
- Formatting of assumptions and environments is a house-style choice, not a CFP rule;
the CFP-level constraints remain the 12-page body and the single PDF.
Output format
[Re-derivability verdict] re-derivable / gaps found
[Statement completeness] <theorems needing quantifier or assumption repair>
[Constant ledger] tracked / drift at <displays>
[External-results audit] <citations with unchecked hypotheses>
[Illustration floor] seeds+replications stated / absent / no numerics in paper
1---2name: colt-reproducibility3description: Use when strengthening the reproducibility of a COLT (Conference on Learning Theory) paper, where reproducing means re-deriving — complete proofs, explicit assumptions, tracked constants, correctly invoked external results, self-contained notation — plus seeds and scripts for any numerical illustration the paper carries.4metadata:5 source: https://skillmd.com/skills/brycewang-stanford/colt-reproducibility6---78# COLT Reproducibility910At COLT, reproducibility means a competent reader can *re-derive* every claim from the11PDF alone. There is no reproducibility checklist form in the COLT 2026 CFP (checked122026-07-08) — the venue enforces the property the hard way, through referees who13attempt the re-derivation during review. This skill is the pre-submission audit that14makes that attempt succeed.1516## The re-derivability standard1718A theorem is reproducible when all of the following hold:1920- Its statement is formally complete: every symbol quantified, every assumption named21 in the statement itself or cited by an explicit assumption label, the probability22 space and adversary model unambiguous.23- Its proof exists in full inside the submitted PDF — COLT's unlimited appendix24 removes every excuse for "omitted for lack of space."25- Each proof step is locally checkable: an expert reading line k needs only lines26 1..k-1, cited external results, and standard background — never your unpublished27 intuition.28- Constants and parameter regimes survive the chain: if Theorem 1 needs29 n ≥ C·d·log(1/δ), the reader can trace what C is or where it was declared absolute.3031## Audit table: where re-derivation fails3233| Failure mode | Typical symptom in the PDF | Repair |34|---|---|---|35| Hypothesis smuggling | Proof uses independence never assumed | Add the assumption to the statement, or redo the step |36| External-result misuse | "By [X], ..." where [X] needs bounded support you don't have | Check [X]'s hypotheses; find the right variant or prove a lemma |37| Constant drift | C doubles silently between two displays | Number constants (C_1, C_2, ...) and track them in a ledger |38| Case leakage | "The case d=1 is analogous" when it is not | Write the case or prove the reduction |39| Notation overload | Same symbol for a filtration and a function class | One notation table, enforced globally |40| Silent regime switch | Bound proved for T large, quoted for all T | State the threshold explicitly in the theorem |4142## Assumption bookkeeping pattern4344Give assumptions their own numbered environment and cite them by label everywhere:4546```latex47\newtheorem{assumption}{Assumption}4849\begin{assumption}[Bounded losses]\label{ass:bounded}50For all $t$, the loss $\ell_t$ maps to $[0,1]$.51\end{assumption}5253\begin{theorem}\label{thm:regret}54Under Assumptions~\ref{ass:bounded} and~\ref{ass:oblivious}, the algorithm's55regret satisfies $R_T \le 4\sqrt{T \log K}$ for all $T \ge 1$.56\end{theorem}57```5859The payoff is auditable dependency: a reviewer can grep which theorems rely on60obliviousness, and your rebuttal can answer "is Assumption 2 needed for Theorem 3?"61with a pointer instead of an essay.6263## Numerical illustrations, when present6465Some COLT papers plot a simulated regret curve or a phase transition to illustrate a66bound. The theory community's floor for those figures:6768- Fixed seeds, stated replication counts, and error bars whose meaning the caption69 defines; a noisy single run "consistent with the theory" persuades no one here.70- The simulated regime must be the theorem's regime — matching horizon, dimension, and71 noise assumptions — or the mismatch must be acknowledged as exploratory.72- The generating script should be one dependency-light file, releasable after73 acceptance; during review, describe the procedure precisely in the appendix since no74 upload channel exists.75- Never let an illustration silently extend the claim ("the bound appears to hold for76 heavy tails too") without labeling it as conjecture.7778## Vignette: the spine of a lower-bound paper7980Consider a submission whose main result is a $\Omega(\sqrt{TK})$ lower bound via a81new instance family. Its re-derivability spine, in the order a referee will attack it:8283- the instance family's construction, with every distribution parameter explicit and84 the randomization protocol (oblivious vs. adaptive) named;85- the information-theoretic step (say, a KL-divergence calculation) with the exact86 divergence bound displayed, not cited as "standard";87- the reduction from learner performance to the divergence quantity, where hypothesis88 smuggling most often hides (does the argument secretly assume deterministic89 learners? say so or generalize);90- the final optimization over instance parameters, with the maximizing choice written91 out — "choosing ε appropriately" is where constants go to die.9293A referee who can walk this spine without leaving the PDF marks correctness resolved;94each externalized step converts into a review question, and three review questions95into a reject.9697## Pre-submission re-derivation drill98991. Print the numbered-statement list (all definitions, assumptions, lemmas, theorems).1002. For each, a non-author coauthor answers: can I state precisely what this claims,101 including quantifiers, without reading the proof? Rewrite until yes.1023. Verify proofs in dependency order, marking each line verified/unverified; the103 `colt-artifact-evaluation` skill's ledger format works here.1044. Re-check every external citation against the cited source's actual hypotheses —105 allocate real time; this is where careful papers die.1065. Reconcile body sketches against appendix proofs: a sketch that describes an older107 proof strategy than the appendix executes reads as a gap to a referee.108109## Cycle-volatility warnings110111- If a future COLT cycle adds any checklist, code policy, or supplementary channel,112 the current CFP announces it; the 2026 cycle had none (待核实 in later cycles).113- The 12-page body and single-PDF rules that shape where proofs live are the 2026114 formulation; re-read the live CFP before restructuring a paper around them.115- Formatting of assumptions and environments is a house-style choice, not a CFP rule;116 the CFP-level constraints remain the 12-page body and the single PDF.117118## Output format119120```text121[Re-derivability verdict] re-derivable / gaps found122[Statement completeness] <theorems needing quantifier or assumption repair>123[Constant ledger] tracked / drift at <displays>124[External-results audit] <citations with unchecked hypotheses>125[Illustration floor] seeds+replications stated / absent / no numerics in paper126```