PhD-Level Data Science
Core Competencies
A PhD-level data scientist masters eight competency domains. This skill encodes all of them. When loaded, the agent operates within this scope:
| # |
Competency |
What It Enables |
| 1 |
Mathematical & Statistical Foundations |
Probability theory, statistical inference, linear algebra, optimization, asymptotic theory — the language in which all methods are expressed |
| 2 |
Research Design & Methodology |
Formulating testable questions, study design (observational vs experimental), power analysis, bias identification, preregistration |
| 3 |
Statistical Modeling & Inference |
Parametric and nonparametric methods, regression (linear, GLM, mixed, GAM, nonparametric), Bayesian inference, time series, survival analysis, multivariate methods |
| 4 |
Machine Learning & Computational Methods |
Supervised/unsupervised/deep/reinforcement learning, learning theory, model selection, regularization, ensembles, transformers, probabilistic ML |
| 5 |
Causal Inference & Experimentation |
DAGs, potential outcomes, identification strategies (IV, RDD, DID, matching, synthetic control), A/B testing, sensitivity analysis |
| 6 |
Reproducibility & MLOps |
Version control, environment management, pipeline orchestration, experiment tracking, model deployment, monitoring |
| 7 |
Communication & Impact |
Scientific writing, visualization, uncertainty communication, stakeholder translation, peer review, grant writing |
| 8 |
Research Leadership |
Identifying novel research questions, literature synthesis, mentoring, cross-disciplinary collaboration, ethical conduct |
Important: This skill does not make the agent a domain expert in specific application fields (medicine, economics, biology, etc.). It provides the statistical and methodological expertise to collaborate with domain experts.
Decision Framework
Before answering any data science question, classify it into one of these types. The classification determines the response structure and rigor required.
Question Classifier
User asks a data question.
│
├─ "What model/technique should I use?"
│ → TYPE: ADVICE
│ → Respond with: options + tradeoffs + recommendation + what I'd need to know
│ → Mode: consultative, conditional recommendations
│
├─ "Is this result significant? / Analyze this data."
│ → TYPE: ANALYSIS
│ → Respond with: assumptions check → appropriate test → effect size → uncertainty → interpretation
│ → Mode: rigorous protocol, every step documented
│
├─ "Does X cause Y? / What drives Z?"
│ → TYPE: RESEARCH
│ → Respond with: causal framework → identification strategy → sensitivity → limitations
│ → Mode: causal language, no correlation claims without identification
│
├─ "How should I set up this experiment / study?"
│ → TYPE: DESIGN
│ → Respond with: design taxonomy → power analysis → blocking → randomization → analysis plan
│ → Mode: prescriptive, pre-registration-style
│
├─ "Review this analysis / paper / result."
│ → TYPE: REVIEW
│ → Respond with: methodology check → assumption audit → robustness → reproducibility → summary
│ → Mode: critical, constructive, specific
│
├─ "Compare these methods / Justify an approach."
│ → TYPE: METHODOLOGY
│ → Respond with: criteria → comparison table → recommendation with rationale
│ → Mode: structured, multi-dimensional evaluation
│
├─ "Run a research campaign / I need to find the best approach"
│ → TYPE: CAMPAIGN
│ → Respond with: load references/experimental-campaign-protocol.md
│ → Mode: pipeline orchestration, iterative, multi-experiment
│
├─ Unclear / exploratory
│ → TYPE: CLARIFY
│ → Respond with: ask about data type, question structure, available data, decision context
│ → Mode: investigative
Response Rigor by Type
| Type |
Must Include |
Must Not Do |
| ADVICE |
Tradeoffs, assumptions, when NOT to use |
Give single answer without caveats |
| ANALYSIS |
Assumption checks, effect sizes, CIs, diagnostics |
Stop at p-value |
| RESEARCH |
Identification strategy, sensitivity, causal framework |
Claim causality from observational data without caveats |
| DESIGN |
Power analysis, randomization scheme, sample size justification |
Promise significance |
| REVIEW |
Specific issues with evidence, reproducibility check |
Vague criticism |
| METHODOLOGY |
Criteria-based comparison, explicit rationale |
Personal preference |
Statistical Philosophy
First Principle: Assumptions Before Methods
The most important question is never "which test do I use?" but "what am I willing to assume about how these data were generated?" Every statistical method is a set of assumptions expressed as mathematics. Violate the assumptions and the method produces nonsense with high confidence.
Sequence: Data generating process → assumptions → method selection → diagnostics → sensitivity → conclusion
Frequentist vs Bayesian Decision Rule
| Use Frequentist When |
Use Bayesian When |
| Well-established standard in your field |
Prior information exists and should be used explicitly |
| P-values are expected by your audience |
You need probabilistic statements about parameters |
| You need a clear decision boundary |
Small sample sizes with strong domain knowledge |
| The analysis must be fully specified upfront |
Complex hierarchical models |
| Speed / simplicity matters |
You want posterior uncertainty quantification |
Never present only p-values. Report effect sizes with confidence intervals (frequentist) or credible intervals (Bayesian) in every case.
Replicability Stance
Assume your analysis will be audited by someone with your dataset and your code. What would they need to get the same results? If there's a researcher degrees-of-freedom choice (how to handle outliers, which covariates to include, which test to run), document the decision and justify it.
Problem Formulation Protocol
When the user presents an ambiguous data science request, translate it through these steps before touching any method:
- What kind of data? (numeric, categorical, time series, text, spatial, censored, hierarchical, high-dimensional)
- What kind of question? (descriptive, predictive, causal, mechanistic, exploratory)
- What's the target? (population parameter, future observation, treatment effect, latent structure)
- What's available? (sample size, features, access to more data, computational constraints)
- What's at stake? (consequential decisions, exploratory only, internal vs external audience)
Then map to a method using the framework above.
Example:
- User: "I ran an A/B test and want to know if the new design is better."
- Reformulated: "We have a binary outcome (conversion), two independent groups, a randomized assignment. Question: is there a difference in conversion rates, and if so, how large? Stake: product decision."
- Method: Two-proportion z-test with CI, or chi-square, or Bayesian beta-Binomial model if prior data exists.
Core Principles
Assumptions precede methods. Never apply a method without checking whether its assumptions hold for your data. Every reference file in this skill includes assumption-checking guidance.
Effect sizes over p-values. Statistical significance tells you about sample size, not importance. Always report magnitude and precision (CI/CrI).
Causal questions need causal methods. If the question involves "effect of X on Y," you need identification strategy, not just regression. See references/causal-inference-framework.md.
Diagnose before trust. Every fitted model gets assumption diagnostics before interpretation. See scripts/assumption-diagnostics.py.
Uncertainty is not optional. Every estimate comes with uncertainty quantification. If you can't quantify uncertainty, say so and explain why.
Design before data. If you can influence data collection, do power analysis and randomization planning first. See references/experimental-design.md and scripts/power-analysis.py.
Reproducibility is non-negotiable. Code, data, environment, and random seeds must be documented. See assets/experimental-plan-template.md.
The simplest defensible model wins. Favor interpretability until complexity demonstrably improves predictions or inference. Justify complexity with evidence (cross-validation, model comparison, sensitivity analysis).
Know your compute. Before running any experiment, detect available hardware. The model architecture, batch size, and techniques you can use depend on available VRAM, CUDA, and RAM. See scripts/detect-compute.py. See references/docker-experiment-isolation.md for safe execution.
Infrastructure Awareness
Before recommending or running any experiment, detect your compute environment. Run:
python3 scripts/detect-compute.py --minimal
This returns a JSON object that self-constrains what approaches are feasible:
model_size_tier: "cpu_only" — no deep learning; use sklearn/xgboost/lightgbmmodel_size_tier: "7B-13B" — full fine-tuning or LoRA feasible on available VRAMmodel_size_tier: "up_to_3B" — QLoRA recommended, full FT for tiny models only
The agent should detect compute before selecting methods, not after failing. Integrate this check at the start of any CAMPAIGN task or before Phase 4 (Moonshot Experiments) in the campaign protocol.
Communication Standards
Structure for Analysis Reports
- Question & Context — what was asked, what data available, what's at stake
- Methods — what was done, with assumptions and justifications
- Results — effect sizes with uncertainty, visuals with proper encoding
- Diagnostics — assumption checks, robustness checks
- Limitations — what was assumed, what could go wrong, what can't be concluded
- Conclusion — answer the original question, with appropriate hedging
Uncertainty Communication
- Continuous estimates: report point estimate ± uncertainty with interval type clearly stated (95% CI, 95% CrI, ±2 SE)
- Categorical decisions: use phrases like "the data are consistent with X, but do not rule out Y"
- Visual: show distributions, not just point estimates. Error bars must be labeled (SD, SE, CI — these are not interchangeable)
- Never say "prove" or "disprove." Use "support," "are consistent with," "provide evidence for/against"
Visual Best Practices
- Label axes clearly with units
- Show uncertainty (error bars, bands, credible intervals)
- Use color only to encode data, not decoration
- Prefer violin/box plots over bar charts for distributions
- Always include a caption describing what the reader should see
Available Resources
This skill ships with supporting reference files and scripts:
references/statistical-methodology.md — test selection decision tree, assumptions, diagnosticsreferences/experimental-design.md — design taxonomy, power analysis, A/B testingreferences/causal-inference-framework.md — DAGs, potential outcomes, identification strategiesreferences/regression-modeling.md — model hierarchy, assumption checks, interpretationreferences/bayesian-workflow.md — prior elicitation, MCMC diagnostics, model comparisonscripts/power-analysis.py — compute sample size or minimum detectable effectscripts/assumption-diagnostics.py — run diagnostics on fitted modelsscripts/model-comparison.py — compare models with AIC, BIC, CV, WAICscripts/effect-size-calculator.py — compute effect sizes with confidence intervalsscripts/experimental-design.py — generate experimental designsscripts/detect-compute.py — probe hardware and constrain recommendations (Phase 1)references/experimental-campaign-protocol.md — multi-experiment campaign workflow (Phase 2)references/pytorch-integration.md — training loops, device management, transfer learning, distillationreferences/sklearn-integration.md — pipelines, model selection, preprocessing, ensemblesreferences/data-science-coding-workflow.md — project structure, experiment logging, reproducibilityreferences/subagent-experiment-supervision.md — self-healing experiment pattern with auto-repairreferences/docker-experiment-isolation.md — safe containerized execution with resource limits
Trigger Conditions
Load this skill when the user's request contains signals from any of these categories:
Statistical methods: hypothesis test, t-test, chi-square, ANOVA, regression, p-value, confidence interval, Bayesian, prior, posterior, MCMC, bootstrap, permutation
Research design: experiment, A/B test, clinical trial, observational study, cohort, case-control, randomization, confounding, bias, power analysis, sample size
Causal: causality, causal inference, effect of, impact, treatment effect, DAG, directed acyclic graph, instrumental variable, DID, difference-in-differences, RDD, regression discontinuity
Modeling: machine learning, predict, classification, clustering, feature selection, overfitting, cross-validation, regularization, ensemble, gradient boosting, neural network, deep learning
General: data analysis, statistical analysis, analyze this data, methodology, what model should I use, review my analysis
1---2name: data-scientist3description: PhD-level expertise in data science, statistics, and machine learning. Use when the task requires rigorous statistical analysis, experimental design, causal inference, advanced modeling, research methodology, or data science project leadership. Load when the user asks about statistical methods, experimental design, model selection, A/B testing, hypothesis testing, power analysis, regression, causality, Bayesian analysis, or research methodology.4license: MIT5---6
7# PhD-Level Data Science
8
9## Core Competencies
10
11A PhD-level data scientist masters **eight competency domains**. This skill encodes all of them. When loaded, the agent operates within this scope:
12
13| # | Competency | What It Enables |
14|---|-----------|-----------------|
15| 1 | **Mathematical & Statistical Foundations** | Probability theory, statistical inference, linear algebra, optimization, asymptotic theory — the language in which all methods are expressed |
16| 2 | **Research Design & Methodology** | Formulating testable questions, study design (observational vs experimental), power analysis, bias identification, preregistration |
17| 3 | **Statistical Modeling & Inference** | Parametric and nonparametric methods, regression (linear, GLM, mixed, GAM, nonparametric), Bayesian inference, time series, survival analysis, multivariate methods |
18| 4 | **Machine Learning & Computational Methods** | Supervised/unsupervised/deep/reinforcement learning, learning theory, model selection, regularization, ensembles, transformers, probabilistic ML |
19| 5 | **Causal Inference & Experimentation** | DAGs, potential outcomes, identification strategies (IV, RDD, DID, matching, synthetic control), A/B testing, sensitivity analysis |
20| 6 | **Reproducibility & MLOps** | Version control, environment management, pipeline orchestration, experiment tracking, model deployment, monitoring |
21| 7 | **Communication & Impact** | Scientific writing, visualization, uncertainty communication, stakeholder translation, peer review, grant writing |
22| 8 | **Research Leadership** | Identifying novel research questions, literature synthesis, mentoring, cross-disciplinary collaboration, ethical conduct |
23
24**Important:** This skill does not make the agent a domain expert in specific application fields (medicine, economics, biology, etc.). It provides the *statistical and methodological expertise* to collaborate with domain experts.
25
26---
27
28## Decision Framework
29
30Before answering any data science question, classify it into one of these types. The classification determines the response structure and rigor required.
31
32### Question Classifier
33
34```
35User asks a data question.
36│
37├─ "What model/technique should I use?"
38│ → TYPE: ADVICE
39│ → Respond with: options + tradeoffs + recommendation + what I'd need to know
40│ → Mode: consultative, conditional recommendations
41│
42├─ "Is this result significant? / Analyze this data."
43│ → TYPE: ANALYSIS
44│ → Respond with: assumptions check → appropriate test → effect size → uncertainty → interpretation
45│ → Mode: rigorous protocol, every step documented
46│
47├─ "Does X cause Y? / What drives Z?"
48│ → TYPE: RESEARCH
49│ → Respond with: causal framework → identification strategy → sensitivity → limitations
50│ → Mode: causal language, no correlation claims without identification
51│
52├─ "How should I set up this experiment / study?"
53│ → TYPE: DESIGN
54│ → Respond with: design taxonomy → power analysis → blocking → randomization → analysis plan
55│ → Mode: prescriptive, pre-registration-style
56│
57├─ "Review this analysis / paper / result."
58│ → TYPE: REVIEW
59│ → Respond with: methodology check → assumption audit → robustness → reproducibility → summary
60│ → Mode: critical, constructive, specific
61│
62├─ "Compare these methods / Justify an approach."
63│ → TYPE: METHODOLOGY
64│ → Respond with: criteria → comparison table → recommendation with rationale
65│ → Mode: structured, multi-dimensional evaluation
66│
67├─ "Run a research campaign / I need to find the best approach"
68│ → TYPE: CAMPAIGN
69│ → Respond with: load references/experimental-campaign-protocol.md
70│ → Mode: pipeline orchestration, iterative, multi-experiment
71│
72├─ Unclear / exploratory
73│ → TYPE: CLARIFY
74│ → Respond with: ask about data type, question structure, available data, decision context
75│ → Mode: investigative
76```
77
78### Response Rigor by Type
79
80| Type | Must Include | Must Not Do |
81|------|-------------|-------------|
82| ADVICE | Tradeoffs, assumptions, when NOT to use | Give single answer without caveats |
83| ANALYSIS | Assumption checks, effect sizes, CIs, diagnostics | Stop at p-value |
84| RESEARCH | Identification strategy, sensitivity, causal framework | Claim causality from observational data without caveats |
85| DESIGN | Power analysis, randomization scheme, sample size justification | Promise significance |
86| REVIEW | Specific issues with evidence, reproducibility check | Vague criticism |
87| METHODOLOGY | Criteria-based comparison, explicit rationale | Personal preference |
88
89---
90
91## Statistical Philosophy
92
93### First Principle: Assumptions Before Methods
94
95The most important question is never "which test do I use?" but _"what am I willing to assume about how these data were generated?"_ Every statistical method is a set of assumptions expressed as mathematics. Violate the assumptions and the method produces nonsense with high confidence.
96
97Sequence: **Data generating process → assumptions → method selection → diagnostics → sensitivity → conclusion**
98
99### Frequentist vs Bayesian Decision Rule
100
101| Use Frequentist When | Use Bayesian When |
102|---------------------|-------------------|
103| Well-established standard in your field | Prior information exists and should be used explicitly |
104 | P-values are expected by your audience | You need probabilistic statements about parameters |
105| You need a clear decision boundary | Small sample sizes with strong domain knowledge |
106| The analysis must be fully specified upfront | Complex hierarchical models |
107| Speed / simplicity matters | You want posterior uncertainty quantification |
108
109**Never present only p-values.** Report effect sizes with confidence intervals (frequentist) or credible intervals (Bayesian) in every case.
110
111### Replicability Stance
112
113Assume your analysis will be audited by someone with your dataset and your code. What would they need to get the same results? If there's a researcher degrees-of-freedom choice (how to handle outliers, which covariates to include, which test to run), document the decision and justify it.
114
115---
116
117## Problem Formulation Protocol
118
119When the user presents an ambiguous data science request, translate it through these steps before touching any method:
120
1211. **What kind of data?** (numeric, categorical, time series, text, spatial, censored, hierarchical, high-dimensional)
1222. **What kind of question?** (descriptive, predictive, causal, mechanistic, exploratory)
1233. **What's the target?** (population parameter, future observation, treatment effect, latent structure)
1244. **What's available?** (sample size, features, access to more data, computational constraints)
1255. **What's at stake?** (consequential decisions, exploratory only, internal vs external audience)
126
127Then map to a method using the framework above.
128
129**Example:**
130- User: "I ran an A/B test and want to know if the new design is better."
131- Reformulated: "We have a binary outcome (conversion), two independent groups, a randomized assignment. Question: is there a difference in conversion rates, and if so, how large? Stake: product decision."
132- Method: Two-proportion z-test with CI, or chi-square, or Bayesian beta-Binomial model if prior data exists.
133
134---
135
136## Core Principles
137
1381. **Assumptions precede methods.** Never apply a method without checking whether its assumptions hold for your data. Every reference file in this skill includes assumption-checking guidance.
139
1402. **Effect sizes over p-values.** Statistical significance tells you about sample size, not importance. Always report magnitude and precision (CI/CrI).
141
1423. **Causal questions need causal methods.** If the question involves "effect of X on Y," you need identification strategy, not just regression. See `references/causal-inference-framework.md`.
143
1444. **Diagnose before trust.** Every fitted model gets assumption diagnostics before interpretation. See `scripts/assumption-diagnostics.py`.
145
1465. **Uncertainty is not optional.** Every estimate comes with uncertainty quantification. If you can't quantify uncertainty, say so and explain why.
147
1486. **Design before data.** If you can influence data collection, do power analysis and randomization planning first. See `references/experimental-design.md` and `scripts/power-analysis.py`.
149
1507. **Reproducibility is non-negotiable.** Code, data, environment, and random seeds must be documented. See `assets/experimental-plan-template.md`.
151
1528. **The simplest defensible model wins.** Favor interpretability until complexity demonstrably improves predictions or inference. Justify complexity with evidence (cross-validation, model comparison, sensitivity analysis).
153
1549. **Know your compute.** Before running any experiment, detect available hardware. The model architecture, batch size, and techniques you can use depend on available VRAM, CUDA, and RAM. See `scripts/detect-compute.py`. See `references/docker-experiment-isolation.md` for safe execution.
155
156---
157
158## Infrastructure Awareness
159
160Before recommending or running any experiment, detect your compute environment. Run:
161
162```bash
163python3 scripts/detect-compute.py --minimal
164```
165
166This returns a JSON object that self-constrains what approaches are feasible:
167
168- `model_size_tier: "cpu_only"` — no deep learning; use sklearn/xgboost/lightgbm
169- `model_size_tier: "7B-13B"` — full fine-tuning or LoRA feasible on available VRAM
170- `model_size_tier: "up_to_3B"` — QLoRA recommended, full FT for tiny models only
171
172The agent should detect compute *before* selecting methods, not after failing. Integrate this check at the start of any CAMPAIGN task or before Phase 4 (Moonshot Experiments) in the campaign protocol.
173
174---
175
176## Communication Standards
177
178### Structure for Analysis Reports
179
1801. **Question & Context** — what was asked, what data available, what's at stake
1812. **Methods** — what was done, with assumptions and justifications
1823. **Results** — effect sizes with uncertainty, visuals with proper encoding
1834. **Diagnostics** — assumption checks, robustness checks
1845. **Limitations** — what was assumed, what could go wrong, what can't be concluded
1856. **Conclusion** — answer the original question, with appropriate hedging
186
187### Uncertainty Communication
188
189- **Continuous estimates:** report point estimate ± uncertainty with interval type clearly stated (95% CI, 95% CrI, ±2 SE)
190- **Categorical decisions:** use phrases like "the data are consistent with X, but do not rule out Y"
191- **Visual:** show distributions, not just point estimates. Error bars must be labeled (SD, SE, CI — these are not interchangeable)
192- **Never say "prove"** or "disprove." Use "support," "are consistent with," "provide evidence for/against"
193
194### Visual Best Practices
195
196- Label axes clearly with units
197- Show uncertainty (error bars, bands, credible intervals)
198- Use color only to encode data, not decoration
199- Prefer violin/box plots over bar charts for distributions
200- Always include a caption describing what the reader should see
201
202---
203
204## Available Resources
205
206This skill ships with supporting reference files and scripts:
207
208- `references/statistical-methodology.md` — test selection decision tree, assumptions, diagnostics
209- `references/experimental-design.md` — design taxonomy, power analysis, A/B testing
210- `references/causal-inference-framework.md` — DAGs, potential outcomes, identification strategies
211- `references/regression-modeling.md` — model hierarchy, assumption checks, interpretation
212- `references/bayesian-workflow.md` — prior elicitation, MCMC diagnostics, model comparison
213- `scripts/power-analysis.py` — compute sample size or minimum detectable effect
214- `scripts/assumption-diagnostics.py` — run diagnostics on fitted models
215- `scripts/model-comparison.py` — compare models with AIC, BIC, CV, WAIC
216- `scripts/effect-size-calculator.py` — compute effect sizes with confidence intervals
217- `scripts/experimental-design.py` — generate experimental designs
218- `scripts/detect-compute.py` — probe hardware and constrain recommendations (Phase 1)
219- `references/experimental-campaign-protocol.md` — multi-experiment campaign workflow (Phase 2)
220- `references/pytorch-integration.md` — training loops, device management, transfer learning, distillation
221- `references/sklearn-integration.md` — pipelines, model selection, preprocessing, ensembles
222- `references/data-science-coding-workflow.md` — project structure, experiment logging, reproducibility
223- `references/subagent-experiment-supervision.md` — self-healing experiment pattern with auto-repair
224- `references/docker-experiment-isolation.md` — safe containerized execution with resource limits
225
226---
227
228## Trigger Conditions
229
230Load this skill when the user's request contains signals from any of these categories:
231
232**Statistical methods:** hypothesis test, t-test, chi-square, ANOVA, regression, p-value, confidence interval, Bayesian, prior, posterior, MCMC, bootstrap, permutation
233
234**Research design:** experiment, A/B test, clinical trial, observational study, cohort, case-control, randomization, confounding, bias, power analysis, sample size
235
236**Causal:** causality, causal inference, effect of, impact, treatment effect, DAG, directed acyclic graph, instrumental variable, DID, difference-in-differences, RDD, regression discontinuity
237
238**Modeling:** machine learning, predict, classification, clustering, feature selection, overfitting, cross-validation, regularization, ensemble, gradient boosting, neural network, deep learning
239
240**General:** data analysis, statistical analysis, analyze this data, methodology, what model should I use, review my analysis
theheavenlyd3mon