Research Paper Writing Pipeline
End-to-end pipeline for producing publication-ready ML/AI research papers targeting NeurIPS, ICML, ICLR, ACL, AAAI, and COLM. This skill covers the full research lifecycle: experiment design, execution, monitoring, analysis, paper writing, review, revision, and submission.
This is not a linear pipeline — it is an iterative loop. Results trigger new experiments. Reviews trigger new analysis. The agent must handle these feedback loops.
┌─────────────────────────────────────────────────────────────┐
│ RESEARCH PAPER PIPELINE │
│ │
│ Phase 0: Project Setup ──► Phase 1: Literature Review │
│ │ │ │
│ ▼ ▼ │
│ Phase 2: Experiment Phase 5: Paper Drafting ◄──┐ │
│ Design │ │ │
│ │ ▼ │ │
│ ▼ Phase 6: Self-Review │ │
│ Phase 3: Execution & & Revision ──────────┘ │
│ Monitoring │ │
│ │ ▼ │
│ ▼ Phase 7: Submission │
│ Phase 4: Analysis ─────► (feeds back to Phase 2 or 5) │
│ │
└─────────────────────────────────────────────────────────────┘
When To Use This Skill
Use this skill when:
- Starting a new research paper from an existing codebase or idea
- Designing and running experiments to support paper claims
- Writing or revising any section of a research paper
- Preparing for submission to a specific conference
- Responding to reviews with additional experiments or revisions
- Converting a paper between conference formats
Core Philosophy
- Be proactive. Deliver complete drafts, not questions. Scientists are busy — produce something concrete they can react to, then iterate.
- Never hallucinate citations. AI-generated citations have ~40% error rate. Always fetch programmatically. Mark unverifiable citations as
[CITATION NEEDED]. - Paper is a story, not a collection of experiments. Every paper needs one clear contribution stated in a single sentence. If you can't do that, the paper isn't ready.
- Experiments serve claims. Every experiment must explicitly state which claim it supports. Never run experiments that don't connect to the paper's narrative.
- Commit early, commit often. Every completed experiment batch, every paper draft update — commit with descriptive messages. Git log is the experiment history.
Proactivity and Collaboration
Default: Be proactive. Draft first, ask with the draft.
| Confidence Level | Action |
|---|---|
| High (clear repo, obvious contribution) | Write full draft, deliver, iterate on feedback |
| Medium (some ambiguity) | Write draft with flagged uncertainties, continue |
| Low (major unknowns) | Ask 1-2 targeted questions via clarify, then draft |
| Section | Draft Autonomously? | Flag With Draft |
|---|---|---|
| Abstract | Yes | "Framed contribution as X — adjust if needed" |
| Introduction | Yes | "Emphasized problem Y — correct if wrong" |
| Methods | Yes | "Included details A, B, C — add missing pieces" |
| Experiments | Yes | "Highlighted results 1, 2, 3 — reorder if needed" |
| Related Work | Yes | "Cited papers X, Y, Z — add any I missed" |
Block for input only when: target venue unclear, multiple contradictory framings, results seem incomplete, explicit request to review first.
Phase 0: Project Setup
Goal: Establish the workspace, understand existing work, identify the contribution.
Step 0.1: Explore the Repository
# Understand project structure
ls -la
find . -name "*.py" | head -30
find . -name "*.md" -o -name "*.txt" | xargs grep -l -i "result\|conclusion\|finding"
Look for:
README.md— project overview and claimsresults/,outputs/,experiments/— existing findingsconfigs/— experimental settings.bibfiles — existing citations- Draft documents or notes
Step 0.2: Organize the Workspace
Establish a consistent workspace structure:
workspace/
paper/ # LaTeX source, figures, compiled PDFs
experiments/ # Experiment runner scripts
code/ # Core method implementation
results/ # Raw experiment results (auto-generated)
tasks/ # Task/benchmark definitions
human_eval/ # Human evaluation materials (if needed)
Step 0.3: Set Up Version Control
git init # if not already
git remote add origin <repo-url>
git checkout -b paper-draft # or main
Git discipline: Every completed experiment batch gets committed with a descriptive message. Example:
Add Monte Carlo constrained results (5 runs, Sonnet 4.6, policy memo task)
Add Haiku baseline comparison: autoreason vs refinement baselines at cheap model tier
Step 0.4: Identify the Contribution
Before writing anything, articulate:
- The What: What is the single thing this paper contributes?
- The Why: What evidence supports it?
- The So What: Why should readers care?
Propose to the scientist: "Based on my understanding, the main contribution is: [one sentence]. The key results show [Y]. Is this the framing you want?"
Step 0.5: Create a TODO List
Use the todo tool to create a structured project plan:
Research Paper TODO:
- [ ] Define one-sentence contribution
- [ ] Literature review (related work + baselines)
- [ ] Design core experiments
- [ ] Run experiments
- [ ] Analyze results
- [ ] Write first draft
- [ ] Self-review (simulate reviewers)
- [ ] Revise based on review
- [ ] Submission prep
Update this throughout the project. It serves as the persistent state across sessions.
Phase 1: Literature Review
Goal: Find related work, identify baselines, gather citations.
Step 1.1: Identify Seed Papers
Start from papers already referenced in the codebase:
# Via terminal:
grep -r "arxiv\|doi\|cite" --include="*.md" --include="*.bib" --include="*.py"
find . -name "*.bib"
Step 1.2: Search for Related Work
Load the arxiv skill for structured paper discovery: skill_view("arxiv"). It provides arXiv REST API search, Semantic Scholar citation graphs, author profiles, and BibTeX generation.
Use web_search for broad discovery, web_extract for fetching specific papers:
# Via web_search:
web_search("[main technique] + [application domain] site:arxiv.org")
web_search("[baseline method] comparison ICML NeurIPS 2024")
# Via web_extract (for specific papers):
web_extract("https://arxiv.org/abs/2303.17651")
Additional search queries to try:
Search queries:
- "[main technique] + [application domain]"
- "[baseline method] comparison"
- "[problem name] state-of-the-art"
- Author names from existing citations
Recommended: Install Exa MCP for real-time academic search:
Codex mcp add exa -- npx -y mcp-remote "https://mcp.exa.ai/mcp"
Step 1.3: Verify Every Citation
NEVER generate BibTeX from memory. ALWAYS fetch programmatically.
For each citation, follow the mandatory 5-step process:
Citation Verification (MANDATORY per citation):
1. SEARCH → Query Semantic Scholar or Exa MCP with specific keywords
2. VERIFY → Confirm paper exists in 2+ sources (Semantic Scholar + arXiv/CrossRef)
3. RETRIEVE → Get BibTeX via DOI content negotiation (programmatically, not from memory)
4. VALIDATE → Confirm the claim you're citing actually appears in the paper
5. ADD → Add verified BibTeX to bibliography
If ANY step fails → mark as [CITATION NEEDED], inform scientist
# Fetch BibTeX via DOI
import requests
def doi_to_bibtex(doi: str) -> str:
response = requests.get(
f"https://doi.org/{doi}",
headers={"Accept": "application/x-bibtex"}
)
response.raise_for_status()
return response.text
If you cannot verify a citation:
\cite{PLACEHOLDER_author2024_verify_this} % TODO: Verify this citation exists
Always tell the scientist: "I've marked [X] citations as placeholders that need verification."
See references/citation-workflow.md for complete API documentation and the full CitationManager class.
Step 1.4: Organize Related Work
Group papers by methodology, not paper-by-paper:
Good: "One line of work uses X's assumption [refs] whereas we use Y's assumption because..." Bad: "Smith et al. introduced X. Jones et al. introduced Y. We combine both."
Phase 2: Experiment Design
Goal: Design experiments that directly support paper claims. Every experiment must answer a specific question.
Step 2.1: Map Claims to Experiments
Create an explicit mapping:
| Claim | Experiment | Expected Evidence |
|---|---|---|
| "Our method outperforms baselines" | Main comparison (Table 1) | Win rate, statistical significance |
| "Effect is larger for weaker models" | Model scaling study | Monotonic improvement curve |
| "Convergence requires scope constraints" | Constrained vs unconstrained | Convergence rate comparison |
Rule: If an experiment doesn't map to a claim, don't run it.
Step 2.2: Design Baselines
Strong baselines are what separates accepted papers from rejected ones. Reviewers will ask: "Did they compare against X?"
Standard baseline categories:
- Naive baseline: Simplest possible approach
- Strong baseline: Best known existing method
- Ablation baselines: Your method minus one component
- Compute-matched baselines: Same compute budget, different allocation
Step 2.3: Define Evaluation Protocol
Before running anything, specify:
- Metrics: What you're measuring, direction symbols (higher/lower better)
- Aggregation: How results are combined across runs/tasks
- Statistical tests: What tests will establish significance
- Sample sizes: How many runs/problems/tasks
Step 2.4: Write Experiment Scripts
Follow these patterns from successful research pipelines:
Incremental saving — save results after each step for crash recovery:
# Save after each problem/task
result_path = f"results/{task}/{strategy}/result.json"
if os.path.exists(result_path):
continue # Skip already-completed work
# ... run experiment ...
with open(result_path, 'w') as f:
json.dump(result, f, indent=2)
Artifact preservation — save all intermediate outputs:
results/<experiment>/
<task>/
<strategy>/
final_output.md # Final result
history.json # Full trajectory
pass_01/ # Per-iteration artifacts
version_a.md
version_b.md
critic.md
Separation of concerns — keep generation, evaluation, and visualization separate:
run_experiment.py # Core experiment runner
run_baselines.py # Baseline comparison
run_comparison_judge.py # Blind evaluation
analyze_results.py # Statistical analysis
make_charts.py # Visualization
See references/experiment-patterns.md for complete design patterns, cron monitoring, and error recovery.
Phase 3: Experiment Execution & Monitoring
Goal: Run experiments reliably, monitor progress, recover from failures.
Step 3.1: Launch Experiments
Use nohup for long-running experiments:
nohup python run_experiment.py --config config.yaml > logs/experiment_01.log 2>&1 &
echo $! # Record the PID
Parallel execution: Run independent experiments simultaneously, but be aware of API rate limits. 4+ concurrent experiments on the same API will slow each down.
Step 3.2: Set Up Monitoring (Cron Pattern)
For long-running experiments, set up periodic status checks. The cron prompt should follow this template:
Monitor Prompt Template:
1. Check if process is still running: ps aux | grep <pattern>
2. Read last 30 lines of log: tail -30 <logfile>
3. Check for completed results: ls <result_dir>
4. If results exist, read and report: cat <result_file>
5. If all done, commit: git add -A && git commit -m "<descriptive message>" && git push
6. Report in structured format (tables with key metrics)
7. Answer the key analytical question for this experiment
Silent mode: If nothing has changed since the last check, respond with [SILENT] to suppress notification to the user. Only report when there's news.
Step 3.3: Handle Failures
Common failure modes and recovery:
| Failure | Detection | Recovery |
|---|---|---|
| API rate limit / credit exhaustion | 402/429 errors in logs | Wait, then re-run (scripts skip completed work) |
| Process crash | PID gone, incomplete results | Re-run from last checkpoint |
| Timeout on hard problems | Process stuck, no log progress | Kill and skip, note in results |
| Wrong model ID | Errors referencing model name | Fix ID and re-run |
Key: Scripts should always check for existing results and skip completed work. This makes re-runs safe and efficient.
Step 3.4: Commit Completed Results
After each experiment batch completes:
git add -A
git commit -m "Add <experiment name>: <key finding in 1 line>"
git push
Phase 4: Result Analysis
Goal: Extract findings, compute statistics, identify the story.
Step 4.1: Aggregate Results
Write analysis scripts that:
- Load all result files from a batch
- Compute per-task and aggregate metrics
- Generate summary tables
# Standard analysis pattern
import json, os
from pathlib import Path
results = {}
for result_file in Path("results/").rglob("result.json"):
data = json.loads(result_file.read_text())
strategy = result_file.parent.name
task = result_file.parent.parent.name
results.setdefault(strategy, {})[task] = data
# Compute aggregate metrics
for strategy, tasks in results.items():
scores = [t["score"] for t in tasks.values()]
print(f"{strategy}: mean={np.mean(scores):.1f}, std={np.std(scores):.1f}")
Step 4.2: Statistical Significance
Always compute:
- Error bars: Standard deviation or standard error, specify which
- Confidence intervals: 95% CI for key results
- Pairwise tests: McNemar's test for comparing two methods
- Effect sizes: Cohen's d or h for practical significance
See references/experiment-patterns.md for complete implementations of McNemar's test, bootstrapped CIs, and Cohen's h.
Step 4.3: Identify the Story
After analysis, explicitly answer:
- What is the main finding? State it in one sentence.
- What surprised you? Unexpected results often make the best papers.
- What failed? Failed experiments can be the most informative. Honest reporting of failures strengthens the paper.
- What follow-up experiments are needed? Results often raise new questions.
Step 4.4: Create Figures and Tables
Figures:
- Use vector graphics (PDF) for all plots:
plt.savefig('fig.pdf') - Colorblind-safe palettes (Okabe-Ito or Paul Tol)
- Self-contained captions — reader should understand without main text
- No title inside figure — the caption serves this function
Tables:
- Use
booktabsLaTeX package - Bold best value per metric
- Include direction symbols (higher/lower better)
- Consistent decimal precision
\usepackage{booktabs}
\begin{tabular}{lcc}
\toprule
Method & Accuracy $\uparrow$ & Latency $\downarrow$ \\
\midrule
Baseline & 85.2 & 45ms \\
\textbf{Ours} & \textbf{92.1} & 38ms \\
\bottomrule
\end{tabular}
Step 4.5: Decide: More Experiments or Write?
| Situation | Action |
|---|---|
| Core claims supported, results significant | Move to Phase 5 (writing) |
| Results inconclusive, need more data | Back to Phase 2 (design) |
| Unexpected finding suggests new direction | Back to Phase 2 (design) |
| Missing one ablation reviewers will ask for | Run it, then Phase 5 |
| All experiments done but some failed | Note failures, move to Phase 5 |
Iterative Refinement: Strategy Selection
Any output in this pipeline — paper drafts, experiment scripts, analysis — can be iteratively refined. The autoreason research provides empirical evidence for when each refinement strategy works and when it fails. Use this section to choose the right approach.
Quick Decision Table
| Your Situation | Strategy | Why |
|---|---|---|
| Mid-tier model + constrained task | Autoreason | Sweet spot. Generation-evaluation gap is widest. Baselines actively destroy weak model outputs. |
| Mid-tier model + open task | Autoreason with scope constraints added | Add fixed facts, structure, or deliverable to bound the improvement space. |
| Frontier model + constrained task | Autoreason | Wins 2/3 constrained tasks even at frontier. |
| Frontier model + unconstrained task | Critique-and-revise or single pass | Autoreason comes last. Model self-evaluates well enough. |
| Concrete technical task (system design) | Critique-and-revise | Direct find-and-fix loop is more efficient. |
| Template-filling task (one correct structure) | Single pass or conservative | Minimal decision space. Iteration adds no value. |
| Code with test cases | Autoreason (code variant) | Structured analysis of why it failed before fixing. Recovery rate 62% vs 43%. |
| Very weak model (Llama 8B class) | Single pass | Model too weak for diverse candidates. Invest in generation quality. |
The Generation-Evaluation Gap
Core insight: Autoreason's value depends on the gap between a model's generation capability and its self-evaluation capability.
Model Tier │ Generation │ Self-Eval │ Gap │ Autoreason Value
──────────────────┼────────────┼───────────┼────────┼─────────────────
Weak (Llama 8B) │ Poor │ Poor │ Small │ None — can't generate diverse candidates
Mid (Haiku 3.5) │ Decent │ Poor │ LARGE │ MAXIMUM — 42/42 perfect Borda
Mid (Gemini Flash)│ Decent │ Moderate │ Large │ High — wins 2/3
Strong (Sonnet 4) │ Good │ Decent │ Medium │ Moderate — wins 3/5
Frontier (S4.6) │ Excellent │ Good │ Small │ Only with constraints
This gap is structural, not temporary. As costs drop, today's frontier becomes tomorrow's mid-tier. The sweet spot moves but never disappears.
Autoreason Loop (Summary)
Each pass produces three candidates from fresh, isolated agents:
- Critic → finds problems in incumbent A (no fixes)
- Author B → revises A based on critique
- Synthesizer → merges A and B (randomized labels)
- Judge Panel → 3 blind CoT judges rank A, B, AB via Borda count
- Convergence → A wins k=2 consecutive passes → done
Key parameters:
- k=2 convergence (k=1 premature, k=3 too expensive, no quality gain)
- CoT judges always (3x faster convergence)
- Temperature 0.8 authors, 0.3 judges
- Conservative tiebreak: incumbent wins ties
- Every role is a fresh agent with no shared context
Applying to Paper Drafts
When refining the paper itself through autoreason:
- Provide ground truth to the critic: actual experimental data, result JSONs, statistical outputs. Without this, models hallucinate fabricated ablation studies and fake confidence intervals.
- Use 3 working judges minimum: A broken judge parser doesn't add noise — it prevents equilibrium entirely.
- Scope constrain the revision: "Address these specific weaknesses" not "improve the paper."
Failure Modes
| Failure | Detection | Fix |
|---|---|---|
| No convergence (A never wins) | A wins <15% over 20+ passes | Add scope constraints to the task |
| Synthesis drift | Word counts grow unboundedly | Constrain structure and deliverable |
| Degradation below single pass | Baselines score higher than iterated output | Switch to single pass; model may be too weak |
| Overfitting (code) | High public-test pass, low private-test pass | Use structured analysis, not just test feedback |
| Broken judges | Parsing failures reduce panel below 3 | Fix parser before continuing |
See references/autoreason-methodology.md for complete prompts, Borda scoring details, model selection guide, scope constraint design patterns, and compute budget reference.
Phase 5: Paper Drafting
Goal: Write a complete, publication-ready paper.
The Narrative Principle
The single most critical insight: Your paper is not a collection of experiments — it's a story with one clear contribution supported by evidence.
Every successful ML paper centers on what Neel Nanda calls "the narrative": a short, rigorous, evidence-based technical story with a takeaway readers care about.
Three Pillars (must be crystal clear by end of introduction):
| Pillar | Description | Test |
|---|---|---|
| The What | 1-3 specific novel claims | Can you state them in one sentence? |
| The Why | Rigorous empirical evidence | Do experiments distinguish your hypothesis from alternatives? |
| The So What | Why readers should care | Does this connect to a recognized community problem? |
If you cannot state your contribution in one sentence, you don't yet have a paper.
Time Allocation
Spend approximately equal time on each of:
- The abstract
- The introduction
- The figures
- Everything else combined
Why? Most reviewers form judgments before reaching your methods. Readers encounter your paper as: title → abstract → introduction → figures → maybe the rest.
Writing Workflow
Paper Writing Checklist:
- [ ] Step 1: Define the one-sentence contribution
- [ ] Step 2: Draft Figure 1 (core idea or most compelling result)
- [ ] Step 3: Draft abstract (5-sentence formula)
- [ ] Step 4: Draft introduction (1-1.5 pages max)
- [ ] Step 5: Draft methods
- [ ] Step 6: Draft experiments & results
- [ ] Step 7: Draft related work
- [ ] Step 8: Draft conclusion & discussion
- [ ] Step 9: Draft limitations (REQUIRED by all venues)
- [ ] Step 10: Plan appendix (proofs, extra experiments, details)
- [ ] Step 11: Complete paper checklist
- [ ] Step 12: Final review
Step 5.0: Title
The title is the single most-read element of the paper. It determines whether anyone clicks through to the abstract.
Good titles:
- State the contribution or finding: "Autoreason: When Iterative LLM Refinement Works and Why It Fails"
- Highlight a surprising result: "Scaling Data-Constrained Language Models" (implies you can)
- Name the method + what it does: "DPO: Direct Preference Optimization of Language Models"
Bad titles:
- Too generic: "An Approach to Improving Language Model Outputs"
- Too long: anything over ~15 words
- Jargon-only: "Asymptotic Convergence of Iterative Stochastic Policy Refinement" (who is this for?)
Rules:
- Include your method name if you have one (for citability)
- Include 1-2 keywords reviewers will search for
- Avoid colons unless both halves carry meaning
- Test: would a reviewer know the domain and contribution from the title alone?
Step 5.1: Abstract (5-Sentence Formula)
From Sebastian Farquhar (DeepMind):
1. What you achieved: "We introduce...", "We prove...", "We demonstrate..."
2. Why this is hard and important
3. How you do it (with specialist keywords for discoverability)
4. What evidence you have
5. Your most remarkable number/result
Delete generic openings like "Large language models have achieved remarkable success..."
Step 5.2: Figure 1
Figure 1 is the second thing most readers look at (after abstract). Draft it before writing the introduction — it forces you to clarify the core idea.
| Figure 1 Type | When to Use | Example |
|---|---|---|
| Method diagram | New architecture or pipeline | TikZ flowchart showing your system |
| Results teaser | One compelling result tells the whole story | Bar chart: "Ours vs baselines" with clear gap |
| Problem illustration | The problem is unintuitive | Before/after showing failure mode you fix |
| Conceptual diagram | Abstract contribution needs visual grounding | 2x2 matrix of method properties |
Rules: Figure 1 must be understandable without reading any text. The caption alone should communicate the core idea. Use color purposefully — don't just decorate.
Step 5.3: Introduction (1-1.5 pages max)
Must include:
- Clear problem statement
- Brief approach overview
- 2-4 bullet contribution list (max 1-2 lines each in two-column format)
- Methods should start by page 2-3
Step 5.3: Methods
Enable reimplementation:
- Conceptual outline or pseudocode
- All hyperparameters listed
- Architectural details sufficient for reproduction
- Present final design decisions; ablations go in experiments
Step 5.4: Experiments & Results
For each experiment, explicitly state:
- What claim it supports
- How it connects to main contribution
- What to observe: "the blue line shows X, which demonstrates Y"
Requirements:
- Error bars with methodology (std dev vs std error)
- Hyperparameter search ranges
- Compute infrastructure (GPU type, total hours)
- Seed-setting methods
Step 5.5: Related Work
Organize methodologically, not paper-by-paper. Cite generously — reviewers likely authored relevant papers.
Step 5.6: Limitations (REQUIRED)
All major conferences require this. Honesty helps:
- Reviewers are instructed not to penalize honest limitation acknowledgment
- Pre-empt criticisms by identifying weaknesses first
- Explain why limitations don't undermine core claims
Step 5.7: Conclusion & Discussion
Conclusion (required, 0.5-1 page):
- Restate the contribution in one sentence (different wording from abstract)
- Summarize key findings (2-3 sentences, not a list)
- Implications: what does this mean for the field?
- Future work: 2-3 concrete next steps (not vague "we leave X for future work")
Discussion (optional, sometimes combined with conclusion):
- Broader implications beyond immediate results
- Connections to other subfields
- Honest assessment of when the method does and doesn't work
- Practical deployment considerations
Do NOT introduce new results or claims in the conclusion.
Step 5.8: Appendix Strategy
Appendices are unlimited at all major venues and are essential for reproducibility. Structure:
| Appendix Section | What Goes Here |
|---|---|
| Proofs & Derivations | Full proofs too long for main text. Main text can state theorems with "proof in Appendix A." |
| Additional Experiments | Ablations, scaling curves, per-dataset breakdowns, hyperparameter sensitivity |
| Implementation Details | Full hyperparameter tables, training details, hardware specs, random seeds |
| Dataset Documentation | Data collection process, annotation guidelines, licensing, preprocessing |
| Prompts & Templates | Exact prompts used (for LLM-based methods), evaluation templates |
| Human Evaluation | Annotation interface screenshots, instructions given to annotators, IRB details |
| Additional Figures | Per-task breakdowns, trajectory visualizations, failure case examples |
Rules:
- The main paper must be self-contained — reviewers are not required to read appendices
- Never put critical evidence only in the appendix
- Cross-reference: "Full results in Table 5 (Appendix B)" not just "see appendix"
- Use
\appendixcommand, then\section{A: Proofs}etc.
Page Budget Management
When over the page limit:
| Cut Strategy | Saves | Risk |
|---|---|---|
| Move proofs to appendix | 0.5-2 pages | Low — standard practice |
| Condense related work | 0.5-1 page | Medium — may miss key citations |
| Combine tables with subfigures | 0.25-0.5 page | Low — often improves readability |
Use \vspace{-Xpt} sparingly |
0.1-0.3 page | Low if subtle, high if obvious |
| Remove qualitative examples | 0.5-1 page | Medium — reviewers like examples |
| Reduce figure sizes | 0.25-0.5 page | High — figures must remain readable |
Do NOT: reduce font size, change margins, remove required sections (limitations, broader impact), or use \small/\footnotesize for main text.
Writing Style
Sentence-level clarity (Gopen & Swan's 7 Principles):
| Principle | Rule |
|---|---|
| Subject-verb proximity | Keep subject and verb close |
| Stress position | Place emphasis at sentence ends |
| Topic position | Put context first, new info after |
| Old before new | Familiar info → unfamiliar info |
| One unit, one function | Each paragraph makes one point |
| Action in verb | Use verbs, not nominalizations |
| Context before new | Set stage before presenting |
Word choice (Lipton, Steinhardt):
- Be specific: "accuracy" not "performance"
- Eliminate hedging: drop "may" unless genuinely uncertain
- Consistent terminology throughout
- Avoid incremental vocabulary: "develop", not "combine"
Full writing guide with examples: See references/writing-guide.md
Using LaTeX Templates
Always copy the entire template directory first, then write within it.
Template Setup Checklist:
- [ ] Step 1: Copy entire template directory to new project
- [ ] Step 2: Verify template compiles as-is (before any changes)
- [ ] Step 3: Read the template's example content to understand structure
- [ ] Step 4: Replace example content section by section
- [ ] Step 5: Use template macros (check preamble for \newcommand definitions)
- [ ] Step 6: Clean up template artifacts only at the end
Step 1: Copy the Full Template
cp -r templates/neurips2025/ ~/papers/my-paper/
cd ~/papers/my-paper/
ls -la # Should see: main.tex, neurips.sty, Makefile, etc.
Copy the ENTIRE directory, not just the .tex file. Templates include style files (.sty), bibliography styles (.bst), example content, and Makefiles.
Step 2: Verify Template Compiles First
Before making ANY changes:
latexmk -pdf main.tex
# Or manual: pdflatex main.tex && bibtex main && pdflatex main.tex && pdflatex main.tex
If the unmodified template doesn't compile, fix that first (usually missing TeX packages — install via tlmgr install <package>).
Step 3: Keep Template Content as Reference
Don't immediately delete example content. Comment it out and use as formatting reference:
% Template example (keep for reference):
% \begin{figure}[t]
% \centering
% \includegraphics[width=0.8\linewidth]{example-image}
% \caption{Template shows caption style}
% \end{figure}
% Your actual figure:
\begin{figure}[t]
\centering
\includegraphics[width=0.8\linewidth]{your-figure.pdf}
\caption{Your caption following the same style.}
\end{figure}
Step 4: Replace Content Section by Section
Work through systematically: title/authors → abstract → introduction → methods → experiments → related work → conclusion → references → appendix. Compile after each section.
Step 5: Use Template Macros
\newcommand{\method}{YourMethodName} % Consistent method naming
\newcommand{\eg}{e.g.,\xspace} % Proper abbreviations
\newcommand{\ie}{i.e.,\xspace}
Template Pitfalls
| Pitfall | Problem | Solution |
|---|---|---|
Copying only .tex file |
Missing .sty, won't compile |
Copy entire directory |
Modifying .sty files |
Breaks conference formatting | Never edit style files |
| Adding random packages | Conflicts, breaks template | Only add if necessary |
| Deleting template content early | Lose formatting reference | Keep as comments until done |
| Not compiling frequently | Errors accumulate | Compile after each section |
| Raster PNGs for figures | Blurry in paper | Always use vector PDF via savefig('fig.pdf') |
Quick Template Reference
| Conference | Main File | Style File | Page Limit |
|---|---|---|---|
| NeurIPS 2025 | main.tex |
neurips.sty |
9 pages |
| ICML 2026 | example_paper.tex |
icml2026.sty |
8 pages |
| ICLR 2026 | iclr2026_conference.tex |
iclr2026_conference.sty |
9 pages |
| ACL 2025 | acl_latex.tex |
acl.sty |
8 pages (long) |
| AAAI 2026 | aaai2026-unified-template.tex |
aaai2026.sty |
7 pages |
| COLM 2025 | colm2025_conference.tex |
colm2025_conference.sty |
9 pages |
Universal: Double-blind, references don't count, appendices unlimited, LaTeX required.
Templates in templates/ directory. See templates/README.md for compilation setup (VS Code, CLI, Overleaf, other IDEs).
Tables and Figures
Tables — use booktabs for professional formatting:
\usepackage{booktabs}
\begin{tabular}{lcc}
\toprule
Method & Accuracy $\uparrow$ & Latency $\downarrow$ \\
\midrule
Baseline & 85.2 & 45ms \\
\textbf{Ours} & \textbf{92.1} & 38ms \\
\bottomrule
\end{tabular}
Rules:
- Bold best value per metric
- Include direction symbols ($\uparrow$ higher better, $\downarrow$ lower better)
- Right-align numerical columns
- Consistent decimal precision
Figures:
- Vector graphics (PDF, EPS) for all plots and diagrams —
plt.savefig('fig.pdf') - Raster (PNG 600 DPI) only for photographs
- Colorblind-safe palettes (Okabe-Ito or Paul Tol)
- Verify grayscale readability (8% of men have color vision deficiency)
- No title inside figure — the caption serves this function
- Self-contained captions — reader should understand without main text
Conference Resubmission
For converting between venues, see Phase 7 (Submission Preparation) — it covers the full conversion workflow, page-change table, and post-rejection guidance.
Professional LaTeX Preamble
Add these packages to any paper for professional quality. They are compatible with all major conference style files:
% --- Professional Packages (add after conference style file) ---
% Typography
\usepackage{microtype} % Microtypographic improvements (protrusion, expansion)
% Makes text noticeably more polished — always include
% Tables
\usepackage{booktabs} % Professional table rules (\toprule, \midrule, \bottomrule)
\usepackage{siunitx} % Consistent number formatting, decimal alignment
% Usage: \num{12345} → 12,345; \SI{3.5}{GHz} → 3.5 GHz
% Table alignment: S column type for decimal-aligned numbers
% Figures
\usepackage{graphicx} % Include graphics (\includegraphics)
\usepackage{subcaption} % Subfigures with (a), (b), (c) labels
% Usage: \begin{subfigure}{0.48\textwidth} ... \end{subfigure}
% Diagrams and Algorithms
\usepackage{tikz} % Programmable vector diagrams
\usetikzlibrary{arrows.meta, positioning, shapes.geometric, calc, fit, backgrounds}
\usepackage[ruled,vlined]{algorithm2e} % Professional pseudocode
% Alternative: \usepackage{algorithmicx} if template bundles it
% Cross-references
\usepackage{cleveref} % Smart references: \cref{fig:x} → "Figure 1"
% MUST be loaded AFTER hyperref
% Handles: figures, tables, sections, equations, algorithms
% Math (usually included by conference .sty, but verify)
\usepackage{amsmath,amssymb} % AMS math environments and symbols
\usepackage{mathtools} % Extends amsmath (dcases, coloneqq, etc.)
% Colors (for figures and diagrams)
\usepackage{xcolor} % Color management
% Okabe-Ito colorblind-safe palette:
\definecolor{okblue}{HTML}{0072B2}
\definecolor{okorange}{HTML}{E69F00}
\definecolor{okgreen}{HTML}{009E73}
\definecolor{okred}{HTML}{D55E00}
\definecolor{okpurple}{HTML}{CC79A7}
\definecolor{okcyan}{HTML}{56B4E9}
\definecolor{okyellow}{HTML}{F0E442}
Notes:
microtypeis the single highest-impact package for visual quality. It adjusts character spacing at a sub-pixel level. Always include it.siunitxhandles decimal alignment in tables via theScolumn type — eliminates manual spacing.cleverefmust be loaded afterhyperref. Most conference .sty files load hyperref, so put cleveref last.- Check if the conference template already loads any of these (especially
algorithm,amsmath,graphicx). Don't double-load.
siunitx Table Alignment
siunitx makes number-heavy tables significantly more readable:
\begin{tabular}{l S[table-format=2.1] S[table-format=2.1] S[table-format=2.1]}
\toprule
Method & {Accuracy $\uparrow$} & {F1 $\uparrow$} & {Latency (ms) $\downarrow$} \\
\midrule
Baseline & 85.2 & 83.7 & 45.3 \\
Ablation (no X) & 87.1 & 85.4 & 42.1 \\
\textbf{Ours} & \textbf{92.1} & \textbf{90.8} & \textbf{38.7} \\
\bottomrule
\end{tabular}
The S column type auto-aligns on the decimal point. Headers in {} escape the alignment.
Subfigures
Standard pattern for side-by-side figures:
\begin{figure}[t]
\centering
\begin{subfigure}[b]{0.48\textwidth}
\centering
\includegraphics[width=\textwidth]{fig_results_a.pdf}
\caption{Results on Dataset A.}
\label{fig:results-a}
\end{subfigure}
\hfill
\begin{subfigure}[b]{0.48\textwidth}
\centering
\includegraphics[width=\textwidth]{fig_results_b.pdf}
\caption{Results on Dataset B.}
\label{fig:results-b}
\end{subfigure}
\caption{Comparison of our method across two datasets. (a) shows the scaling
behavior and (b) shows the ablation results. Both use 5 random seeds.}
\label{fig:results}
\end{figure}
Use \cref{fig:results} → "Figure 1", \cref{fig:results-a} → "Figure 1a".
Pseudocode with algorithm2e
\begin{algorithm}[t]
\caption{Iterative Refinement with Judge Panel}
\label{alg:method}
\KwIn{Task $T$, model $M$, judges $J_1 \ldots J_n$, convergence threshold $k$}
\KwOut{Final output $A^*$}
$A \gets M(T)$ \tcp*{Initial generation}
$\text{streak} \gets 0$\;
\While{$\text{streak} < k$}{
$C \gets \text{Critic}(A, T)$ \tcp*{Identify weaknesses}
$B \gets M(T, C)$ \tcp*{Revised version addressing critique}
$AB \gets \text{Synthesize}(A, B)$ \tcp*{Merge best elements}
\ForEach{judge $J_i$}{
$\text{rank}_i \gets J_i(\text{shuffle}(A, B, AB))$ \tcp*{Blind ranking}
}
$\text{winner} \gets \text{BordaCount}(\text{ranks})$\;
\eIf{$\text{winner} = A$}{
$\text{streak} \gets \text{streak} + 1$\;
}{
$A \gets \text{winner}$; $\text{streak} \gets 0$\;
}
}
\Return{$A$}\;
\end{algorithm}
TikZ Diagram Patterns
TikZ is the standard for method diagrams in ML papers. Common patterns:
Pipeline/Flow Diagram (most common in ML papers):
\begin{figure}[t]
\centering
\begin{tikzpicture}[
node distance=1.8cm,
box/.style={rectangle, draw, rounded corners, minimum height=1cm,
minimum width=2cm, align=center, font=\small},
arrow/.style={-{Stealth[length=3mm]}, thick},
]
\node[box, fill=okcyan!20] (input) {Input\\$x$};
\node[box, fill=okblue!20, right of=input] (encoder) {Encoder\\$f_\theta$};
\node[box, fill=okgreen!20, right of=encoder] (latent) {Latent\\$z$};
\node[box, fill=okorange!20, right of=latent] (decoder) {Decoder\\$g_\phi$};
\node[box, fill=okred!20, right of=decoder] (output) {Output\\$\hat{x}$};
\draw[arrow] (input) -- (encoder);
\draw[arrow] (encoder) -- (latent);
\draw[arrow] (latent
…(truncated)