# Numerai Research

> End-to-end Numerai research workflow for trying a new idea: design experiments, implement new model types if needed, run scout→scale experiments, write a full experiment.md report with standard plots, and optionally package/upload a Numerai pickle. Use when a user asks to “try/test a new idea”, “run an experiment”, “sweep configs”, “compare model variants”, or otherwise do new Numerai research.

- Skill: `numerai/numerai-research` (Agent Skill)
- Install (CLI): `npx skillmds@latest add numerai/numerai-research`
- Raw SKILL.md: https://api.skillmd.com/api/skills/numerai/numerai-research/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: numerai (https://skillmd.com/u/numerai)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/numerai/numerai-research

---


# Numerai Research

## Overview

This skill is a “meta-workflow” that sequences existing Numerai skills so research requests reliably produce: (1) runnable configs, (2) executed experiments, (3) a full written report + plots, and (4) a deployable pickle when requested.

## Workflow (always follow this order)

### 1) Design the experiment (use numerai-experiment-design)

- Follow the `numerai-experiment-design` skill to:
  - clarify the idea (or run quick scout interpretations if ambiguous)
  - choose baseline + feature set alignment (default ender20 baseline)
  - create an experiment folder under `numerai/agents/experiments/<experiment_name>/`
  - write configs in `configs/`
  - run training via `PYTHONPATH=numerai python3 -m agents.code.modeling --config <config> --output-dir <experiment_dir>`
  - track metrics with BMC as primary (`bmc_mean`, `bmc_last_200_eras`)
  - **iterate in rounds** (typically 4–5 configs per round), and keep going until you hit a plateau (per the experiment-design skill)
  - **scale winners** (bigger feature set and/or full data) before finalizing the best model

### 2) Implement new model types if needed (use numerai-model-implementation)

Only if the idea requires new code (new model wrapper, new fit/predict behavior, etc.):
- Follow the `numerai-model-implementation` skill to add the model type and register it.
- Add at least one smoke-test config and verify the pipeline runs.

### 3) Report the research (use report-research)

After you have iterated through multiple rounds **and** stopped finding improvements (plateau), and after any confirmatory scale runs:
- Follow the `report-research` skill to:
  - write a full `experiment.md` (abstract + methods + results + decisions + next steps)
  - generate the standard `show_experiment` plot(s)
  - link plots and artifacts in the report

### 4) Package and upload (use numerai-model-upload)

If (and only if) the user wants deployment:
- Follow the `numerai-model-upload` skill to create a Numerai-compatible pickle and upload it via the Numerai MCP.
- Remember: only Classic (tournament 8) supports pickle uploads.

## Defaults (unless user specifies otherwise)

- Scout first on downsampled data; scale only winners.
- Run experiments in rounds (4–5 configs per round) and stop only after a plateau + confirmatory scale step.
- Benchmark reference: `v53_lgbm_ender20`.
- Always record corr + BMC metrics and include the standard plot in the report.