AI Researcher
When to Use
- Surveying state-of-the-art models, methods, or benchmarks
- Comparing model families or techniques with fair experimental design
- Designing ablation studies with controlled variables
- Writing research memos or technical reports for stakeholder decisions
- Critiquing methodology in papers or internal experiments
- Planning novel experiments with falsifiable hypotheses
- Reproducing published results and verifying claims
When NOT to Use
- Shipping production LLM features, RAG, or agent systems →
ai-engineer
- Enterprise AI policy, regulation, or risk tiering →
ai-risk-governance
- Adversarial product testing or jailbreak campaigns →
ai-redteam
- Classical ML pipelines, A/B testing, or statistical analysis →
data-scientist
Related skills
| Need |
Skill |
| Production RAG, agents, deployment |
ai-engineer |
| Prompt and agent implementation detail |
prompt-engineer |
| Classical ML and A/B statistics |
data-scientist |
| Governance, regulation, risk registers |
ai-risk-governance |
| Red-team attacks on deployed systems |
ai-redteam |
| Token/context efficiency research |
research-engineer-scientist-tokens |
| Safeguard ML benchmarks and classifiers |
ml-research-engineer-safeguards |
| RL distributed training infrastructure |
ml-systems-engineer-rl-engineering |
Core Workflows
1. Research question framing
- Convert vague ask into falsifiable question
- Define scope: task, data regime, compute budget, timeline
- List baselines that must be beaten or matched
- Specify primary and secondary metrics
- Document assumptions and out-of-scope items
See references/research_framing.md for question templates and hypothesis types.
2. Literature review
Process:
- Search: arXiv, ACL Anthology, OpenReview, major labs' blogs
- Screen by relevance, recency, citation quality
- Extract: problem, method, data, metrics, limitations
- Synthesize themes and open gaps
- Cite primary sources; avoid over-relying on secondary summaries
See references/literature_review.md for screening matrix and synthesis outline.
3. Experimental design
| Element |
Requirement |
| Baselines |
Strong and fair (same data, tuning budget) |
| Ablations |
One change at a time |
| Seeds |
Multiple runs for stochastic methods |
| Stats |
Confidence intervals, not single-point luck |
| Reproducibility |
Config, data version, code commit logged |
See references/experiment_design.md for power analysis pointers and leakage checks.
4. Benchmarking and analysis
- Use public benchmarks when task-aligned; document train/test contamination risk
- Report compute cost (GPU hours) alongside accuracy
- Separate in-distribution vs stress tests
- Visualize failure modes, not only aggregate scores
See references/benchmarking.md for leaderboard caveats and custom eval sets.
5. Research communication
Deliverable types: memo (1–3 pages), technical report, slide deck for decision meeting.
Include: question, method summary, results table, limitations, recommended next step.
See references/research_writing.md for memo structure and peer-review checklist.
When to load references
- Question and hypothesis →
references/research_framing.md
- Literature survey →
references/literature_review.md
- Experiments →
references/experiment_design.md
- Benchmarks →
references/benchmarking.md
- Writing →
references/research_writing.md
1---2name: ai-researcher3description: Guides AI research work—literature reviews, hypothesis formation, experiment design, benchmarking, reproducibility, and synthesis of papers and empirical results for technical decisions. Use when surveying state of the art, comparing models or methods, designing ablation studies, writing research memos, critiquing methodology, or planning novel experiments—not for shipping production LLM features (ai-engineer), enterprise AI policy (ai-risk-governance), or adversarial product testing (ai-redteam). Token efficiency experiments and tokens-to-success benchmarks: research-engineer-scientist-tokens. Safety classifier and harm-benchmark research: ml-research-engineer-safeguards. RL training systems engineering: ml-systems-engineer-rl-engineering.4---56# AI Researcher78## When to Use910- Surveying state-of-the-art models, methods, or benchmarks11- Comparing model families or techniques with fair experimental design12- Designing ablation studies with controlled variables13- Writing research memos or technical reports for stakeholder decisions14- Critiquing methodology in papers or internal experiments15- Planning novel experiments with falsifiable hypotheses16- Reproducing published results and verifying claims1718## When NOT to Use1920- Shipping production LLM features, RAG, or agent systems → `ai-engineer`21- Enterprise AI policy, regulation, or risk tiering → `ai-risk-governance`22- Adversarial product testing or jailbreak campaigns → `ai-redteam`23- Classical ML pipelines, A/B testing, or statistical analysis → `data-scientist`2425## Related skills2627| Need | Skill |28|---|---|29| Production RAG, agents, deployment | `ai-engineer` |30| Prompt and agent implementation detail | `prompt-engineer` |31| Classical ML and A/B statistics | `data-scientist` |32| Governance, regulation, risk registers | `ai-risk-governance` |33| Red-team attacks on deployed systems | `ai-redteam` |34| Token/context efficiency research | `research-engineer-scientist-tokens` |35| Safeguard ML benchmarks and classifiers | `ml-research-engineer-safeguards` |36| RL distributed training infrastructure | `ml-systems-engineer-rl-engineering` |3738## Core Workflows3940### 1. Research question framing41421. Convert vague ask into falsifiable question432. Define scope: task, data regime, compute budget, timeline443. List baselines that must be beaten or matched454. Specify primary and secondary metrics465. Document assumptions and out-of-scope items4748**See `references/research_framing.md` for question templates and hypothesis types.**4950### 2. Literature review5152**Process:**53541. Search: arXiv, ACL Anthology, OpenReview, major labs' blogs552. Screen by relevance, recency, citation quality563. Extract: problem, method, data, metrics, limitations574. Synthesize themes and open gaps585. Cite primary sources; avoid over-relying on secondary summaries5960**See `references/literature_review.md` for screening matrix and synthesis outline.**6162### 3. Experimental design6364| Element | Requirement |65|---|---|66| Baselines | Strong and fair (same data, tuning budget) |67| Ablations | One change at a time |68| Seeds | Multiple runs for stochastic methods |69| Stats | Confidence intervals, not single-point luck |70| Reproducibility | Config, data version, code commit logged |7172**See `references/experiment_design.md` for power analysis pointers and leakage checks.**7374### 4. Benchmarking and analysis7576- Use public benchmarks when task-aligned; document train/test contamination risk77- Report compute cost (GPU hours) alongside accuracy78- Separate in-distribution vs stress tests79- Visualize failure modes, not only aggregate scores8081**See `references/benchmarking.md` for leaderboard caveats and custom eval sets.**8283### 5. Research communication8485**Deliverable types:** memo (1–3 pages), technical report, slide deck for decision meeting.8687Include: question, method summary, results table, limitations, recommended next step.8889**See `references/research_writing.md` for memo structure and peer-review checklist.**9091## When to load references9293- **Question and hypothesis** → `references/research_framing.md`94- **Literature survey** → `references/literature_review.md`95- **Experiments** → `references/experiment_design.md`96- **Benchmarks** → `references/benchmarking.md`97- **Writing** → `references/research_writing.md`
daemon-blockint-tech