dingxingdi Skill: experiment-driven-model-improvement
1. Capability Definition & Real Case
- Professional Definition: The ability to iteratively improve an existing machine-learning codebase by reading starter files, modifying model or training code, executing experiments, interpreting metrics, and selecting better-performing variants under explicit evaluation constraints.
- Dimension Hierarchy: Data and ML Workflow Engineering->Machine Learning Engineering->experiment-driven-model-improvement
Real Case
[Case 1]
- Initial Environment: A workspace contains train.py, data_description.txt, evaluation_details.txt, and starter code for image classification. The baseline model trains successfully but underperforms the target threshold.
- Real Question: Improve the current training script so the model achieves materially better validation performance without violating the task constraints.
- Real Trajectory: Inspect the baseline architecture and hyperparameters, identify a plausible bottleneck such as learning rate or regularization, edit the script, execute training, compare metrics against the baseline, and iterate on the most promising variant.
- Real Answer: A modified training configuration yields a clear performance improvement over the starter baseline while preserving runnable training and submission behavior.
- Why this demonstrates the capability: This demonstrates experiment-driven improvement because success comes from the full engineering loop: inspect code, change an ML decision, run the experiment, read outputs, and decide whether the intervention improved the measured objective.
[Case 2]
- Initial Environment: A tabular regression task provides starter files, schema notes, and a simple baseline model. The repository includes no severe bugs, but the current feature pipeline is weak.
- Real Question: Revise the existing pipeline to improve the evaluation metric on the provided dataset.
- Real Trajectory: Read the data description, inspect preprocessing and model code, add more suitable features and training settings, run the script to generate a new submission, and compare the resulting score with the baseline.
- Real Answer: The updated pipeline improves the target metric through more effective preprocessing and model configuration than the starter implementation.
- Why this demonstrates the capability: The capability being tested is not generic coding or one-shot model generation; it is iterative ML engineering grounded in observed metrics, existing code structure, and experiment interpretation.
Pipeline Execution Instructions
To synthesize data for this capability, you must strictly follow a 3-phase pipeline. Do not hallucinate steps. Read the corresponding reference file for each phase sequentially:
Phase 1: Environment Exploration Read the exploration guidelines to discover raw knowledge seeds:
references/EXPLORATION.mdPhase 2: Trajectory Selection Once Phase 1 is complete, read the selection criteria to evaluate the trajectory:
references/SELECTION.mdPhase 3: Data Synthesis Once a trajectory passes Phase 2, read the synthesis instructions to generate the final data:
references/SYNTHESIS.md