# Experiment Driven Model Improvement

> Use this skill when the user wants data where the agent must improve an existing ML training setup by editing code and running experiments. Trigger it for requests like 'generate ML debugging and improvement tasks', 'make train.py optimization problems', 'create model tuning workflows with execution', or 'give me data where the agent has to read logs and improve accuracy'. Do not use it for open-ended Kaggle competition work from scratch or for paper-reproduction tasks.

- Skill: `dingxingdi/experiment-driven-model-improvement-2` (Agent Skill, multi-file: 5 files)
- Install (CLI): `npx skillmds@latest add dingxingdi/experiment-driven-model-improvement-2`
- Raw SKILL.md: https://api.skillmd.com/api/skills/dingxingdi/experiment-driven-model-improvement-2/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: dingxingdi (https://skillmd.com/u/dingxingdi)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/dingxingdi/experiment-driven-model-improvement-2

---


# Skill: experiment-driven-model-improvement
## 1. Capability Definition & Real Case
* **Professional Definition**: The ability to iteratively improve an existing machine-learning codebase by reading starter files, modifying model or training code, executing experiments, interpreting metrics, and selecting better-performing variants under explicit evaluation constraints.
* **Dimension Hierarchy**: Data and ML Workflow Engineering->Machine Learning Engineering->experiment-driven-model-improvement
### Real Case
**[Case 1]**
* **Initial Environment**: A workspace contains train.py, data_description.txt, evaluation_details.txt, and starter code for image classification. The baseline model trains successfully but underperforms the target threshold.
* **Real Question**: Improve the current training script so the model achieves materially better validation performance without violating the task constraints.
* **Real Trajectory**: Inspect the baseline architecture and hyperparameters, identify a plausible bottleneck such as learning rate or regularization, edit the script, execute training, compare metrics against the baseline, and iterate on the most promising variant.
* **Real Answer**: A modified training configuration yields a clear performance improvement over the starter baseline while preserving runnable training and submission behavior.
* **Why this demonstrates the capability**: This demonstrates experiment-driven improvement because success comes from the full engineering loop: inspect code, change an ML decision, run the experiment, read outputs, and decide whether the intervention improved the measured objective.
---
**[Case 2]**
* **Initial Environment**: A tabular regression task provides starter files, schema notes, and a simple baseline model. The repository includes no severe bugs, but the current feature pipeline is weak.
* **Real Question**: Revise the existing pipeline to improve the evaluation metric on the provided dataset.
* **Real Trajectory**: Read the data description, inspect preprocessing and model code, add more suitable features and training settings, run the script to generate a new submission, and compare the resulting score with the baseline.
* **Real Answer**: The updated pipeline improves the target metric through more effective preprocessing and model configuration than the starter implementation.
* **Why this demonstrates the capability**: The capability being tested is not generic coding or one-shot model generation; it is iterative ML engineering grounded in observed metrics, existing code structure, and experiment interpretation.

## Pipeline Execution Instructions
To synthesize data for this capability, you must strictly follow a 3-phase pipeline. **Do not hallucinate steps.** Read the corresponding reference file for each phase sequentially:

1. **Phase 1: Environment Exploration**
   Read the exploration guidelines to discover raw knowledge seeds:
   `references/EXPLORATION.md`

2. **Phase 2: Trajectory Selection**
   Once Phase 1 is complete, read the selection criteria to evaluate the trajectory:
   `references/SELECTION.md`

3. **Phase 3: Data Synthesis**
   Once a trajectory passes Phase 2, read the synthesis instructions to generate the final data:
   `references/SYNTHESIS.md`