gabrielmoreira Skill: Statistical and ML Review
Use this skill when doing statistical analysis, experiments, model evaluation, or ML work.
Required Context for Any Analysis
Before starting analysis, clearly state:
- Target variable: What are we predicting or measuring?
- Unit of analysis: What does one row represent?
- Time period: What date range does the data cover?
- Sample definition: What population is included?
- Exclusions: What was filtered out, and why?
- Assumptions: What statistical or business assumptions apply?
- Leakage risks: Could future information leak into training data?
- Evaluation metric: How will success be measured?
- Baseline: What is the simplest comparison point?
Experiments and Causal Inference
- Distinguish correlation from causation explicitly.
- Mention confidence intervals or uncertainty when reporting results.
- Avoid overclaiming — state what the data supports, not what you hope it shows.
- Document the experimental design (A/B test, pre-post, observational, etc.).
Machine Learning
- Separate train, validation, and test sets clearly.
- Avoid leakage — no future data in training, no target leakage in features.
- Document preprocessing — encoding, scaling, imputation, feature engineering.
- Compare against a simple baseline before reporting model performance.
- Consider cross-validation — especially for small datasets where a single split may be unreliable.
- Report limitations — data quality, sample bias, generalizability.
- Version datasets and model artifacts when practical.
Review Checklist
- Is the target variable well-defined?
- Is the unit of analysis clear?
- Are train/test splits time-aware if data is temporal?
- Is there a baseline comparison?
- Are evaluation metrics appropriate for the problem?
- Are limitations documented?
- Is the analysis reproducible?