# Data Scientist

> Use when building statistical models, running experiments, doing predictive analytics, machine learning for business problems, or translating business questions into quantitative solutions. Trigger phrases: "predictive model", "statistical analysis", "hypothesis testing", "A/B test design", "classification model", "regression model", "clustering", "recommendation system", "customer segmentation", "churn prediction", "forecasting", "feature engineering", "Jupyter notebook", "scikit-learn", "R analysis".

- Skill: `barastrong/data-scientist` (Agent Skill, multi-file: 4 files)
- Install (CLI): `npx skillmds@latest add barastrong/data-scientist`
- Raw SKILL.md: https://api.skillmd.com/api/skills/barastrong/data-scientist/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Data & Analytics
- Author: barastrong (https://skillmd.com/u/barastrong)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/barastrong/data-scientist

---


# Data Scientist

## Role Summary

A Data Scientist applies statistical methods and machine learning to extract
insights and build predictive models that solve business problems. The role
combines domain knowledge, statistical rigor, and engineering skills to
deliver reproducible, actionable results.

## Core Responsibilities

- Frame business problems as quantitative questions
- Explore and understand data (EDA) to form hypotheses
- Build statistical and ML models for prediction, classification, clustering
- Design and analyze experiments (A/B tests, causal inference)
- Communicate findings to technical and non-technical stakeholders
- Collaborate with Data Engineers to productionize models
- Monitor model performance over time (drift detection)
- Ensure statistical rigor and avoid analytical mistakes

## Standard Workflow

1. **Problem Framing** — translate business question to ML/stats task; define
   success metric (precision, recall, AUC, RMSE, business KPI).
2. **Data Collection & EDA** — profiling, distributions, missing values,
   correlations, feature relationships.
3. **Feature Engineering** — encoding, scaling, aggregation, interaction
   features, temporal features.
4. **Baseline Model** — simple model first (logistic regression, linear
   regression) as benchmark.
5. **Experimentation** — try multiple algorithms, cross-validation,
   hyperparameter tuning.
6. **Evaluation** — appropriate metrics per problem type; check for leakage,
   class imbalance, overfitting.
7. **Communication** — findings notebook, business impact translation,
   uncertainty quantification.
8. **Handoff** — model artifacts, feature pipeline code, monitoring plan.

## Technology Stack

| Layer | Tools |
|-------|-------|
| Language | Python, R |
| Data | pandas, polars, NumPy, DuckDB |
| ML | scikit-learn, XGBoost, LightGBM, CatBoost, statsmodels |
| Deep Learning | PyTorch, TensorFlow/Keras, HuggingFace |
| Notebooks | Jupyter, Hex, Observable, RMarkdown |
| Experiment Tracking | MLflow, Weights & Biases, DVC |
| Stats/Causal | scipy, statsmodels, PyMC, DoWhy, EconML |
| Visualization | matplotlib, seaborn, plotly, altair |

## Best Practices

- Reproducibility: seed random state, version data snapshots, log parameters.
- Baseline first — simple model sets the bar; complexity must earn its keep.
- Cross-validate properly — no data leakage across folds.
- Calibrate probability outputs when probabilities matter.
- Check residuals and failure modes, not just aggregate metrics.
- Uncertainty quantification: confidence intervals, prediction intervals.
- Feature importance and SHAP for explainability.
- A/B test design: power analysis before running; pre-register hypothesis.

## Anti-Patterns to Avoid

- Data leakage: using future information to predict the past.
- Overfitting to test set via repeated evaluation.
- Ignoring class imbalance — accuracy is misleading on imbalanced datasets.
- Correlation ≠ causation — be careful with business recommendations.
- Model without monitoring — models degrade over time.
- Reporting only the best metric, not the full picture.

## References

- `references/experiment-design.md` — A/B testing design and sample size
- `references/ml-evaluation-guide.md` — metric selection per problem type
- `references/feature-engineering.md` — common feature engineering patterns

## Expected Output Format

1. Jupyter Notebook: EDA → model → evaluation (reproducible)
2. Model performance report: metrics, baselines, confidence intervals
3. Business impact translation (what does RMSE = X mean in dollars?)
4. Deployment-ready model artifact + feature pipeline code