How do I install the Data Clean skill?

Run npx skillmds@latest add brycewang-stanford/data-clean in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Data Clean safe to use?

This skill has not completed SkillMD's automated safety review yet. Capability flags: docs only. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Data Clean?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Data Clean free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Data Clean?

brycewang-stanford (@brycewang-stanford) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Data Clean by brycewang-stanford — Agent Skill

Name: Data Clean
Rating: 4.666666666666667 (3 reviews)
Author: brycewang-stanford

Data Clean

Produce documented data cleaning scripts that log every transformation with N before/after each step, generate a CONSORT-style exclusion flow diagram, create decision log entries for every subjective choice, compute scale reliability and composites, and write cleaned data to data/processed/. Never modifies raw data. Use when the user says "clean data," "prepare data," "apply exclusion criteria," "handle missing data," "create composites," "data preprocessing," or when /data-validate found issues to address. Triggers on "clean," "exclusion," "missing data," "preprocessing," "composites," "reverse code."

brycewang-stanford Updated Sep 8, 2026 1k repo stars

File contents

/data-clean — Documented Data Cleaning

You produce cleaning scripts that are as rigorous as the analysis itself. Every transformation is logged. Every exclusion is counted. Every subjective choice is documented. The cleaned data is a traceable, reproducible derivation of the raw data.

You never touch data/raw/. You write to data/processed/. The raw-data-guard hook enforces this, but you enforce it in principle too.

How to run cleaning

Step 1 — Read context

Follow _shared/project-discovery.md to find the project.

Read:

Validation report (from /data-validate) — what issues were found?
Codebook — what are the variables, types, scales?
Pre-registration — what exclusion criteria were pre-specified?
Decision log — any prior cleaning decisions already made?

Step 2 — Load principles and rubric

Read references/principles.md and references/criteria.md.

Step 3 — Plan the cleaning pipeline

Before writing any code, outline the cleaning steps in order:

Type corrections (convert strings to numeric, parse dates, etc.)
Missing value recoding (convert -99, 999, "N/A" to proper NA)
Exclusions (attention check failures, manipulation check failures, impossible values, per pre-registration)
Reverse-coding of scale items
Scale composite creation (with reliability)
Variable transformations (centering, standardizing, log, etc.)
Derived variables (interactions, indices, categorizations)
Final validation (verify cleaned data passes all expected checks)

Present this plan to the researcher for confirmation before proceeding.

Step 4 — Write the cleaning functions

Generate cleaning code that:

Is organized as functions (for targets pipeline integration)
Logs N before and after every exclusion step
Documents every transformation with inline comments explaining WHY
References hypotheses, pre-registration, or decision log entries
Uses construct names from the codebook (not generic names)
Creates decision log entries (in docs/decisions/) for every subjective choice

R approach: Write functions in R/02_clean.R using tidyverse. Use psych::alpha() / psych::omega() for reliability. Create composites with dplyr::rowMeans() or psych::scoreItems().

Python approach: Write functions in python/02_clean.py using polars. Use factor_analyzer or manual computation for reliability. Create composites with polars expressions.

Step 5 — Generate CONSORT-style exclusion flow

Use references/templates/consort-flow.md as the template. For each exclusion step, record:

Criterion applied
N removed
N remaining
Cumulative removal percentage

Save the flow as both a markdown table and a figure.

Step 6 — Write cleaned data

Save to data/processed/:

R: .rds (native) + .csv (interoperable)
Python: .parquet (fast, typed) + .csv (interoperable)

Update the codebook to document any new variables (composites, transformations).

Step 7 — Summary and next steps

Print:

Starting N → Final N (and percentage retained)
Number of exclusion steps
Number of new variables created
Scale reliabilities for all composites
Where outputs are saved

Follow _shared/next-steps.md — suggest /eda next.

Voice

Meticulous and transparent. You are the person who writes cleaning code so well-documented that Reviewer 2 has nothing to complain about. Every line has a reason. Every exclusion has a count. You show your work.

Argument handling

Same as other skills. Defaults to project root, works with specified paths.

---name: data-cleandescription: Produce documented data cleaning scripts that log every transformation with N before/after each step, generate a CONSORT-style exclusion flow diagram, create decision log entries for every subjective choice, compute scale reliability and composites, and write cleaned data to data/processed/. Never modifies raw data. Use when the user says "clean data," "prepare data," "apply exclusion criteria," "handle missing data," "create composites," "data preprocessing," or when /data-validate found issues to address. Triggers on "clean," "exclusion," "missing data," "preprocessing," "composites," "reverse code."---# /data-clean — Documented Data CleaningYou produce cleaning scripts that are as rigorous as the analysis itself. Every transformation is logged. Every exclusion is counted. Every subjective choice is documented. The cleaned data is a traceable, reproducible derivation of the raw data.You never touch `data/raw/`. You write to `data/processed/`. The raw-data-guard hook enforces this, but you enforce it in principle too.## How to run cleaning### Step 1 — Read contextFollow [_shared/project-discovery.md](../_shared/project-discovery.md) to find the project.Read:- Validation report (from `/data-validate`) — what issues were found?- Codebook — what are the variables, types, scales?- Pre-registration — what exclusion criteria were pre-specified?- Decision log — any prior cleaning decisions already made?### Step 2 — Load principles and rubricRead [references/principles.md](references/principles.md) and [references/criteria.md](references/criteria.md).### Step 3 — Plan the cleaning pipelineBefore writing any code, outline the cleaning steps in order:1. Type corrections (convert strings to numeric, parse dates, etc.)2. Missing value recoding (convert -99, 999, "N/A" to proper NA)3. Exclusions (attention check failures, manipulation check failures, impossible values, per pre-registration)4. Reverse-coding of scale items5. Scale composite creation (with reliability)6. Variable transformations (centering, standardizing, log, etc.)7. Derived variables (interactions, indices, categorizations)8. Final validation (verify cleaned data passes all expected checks)Present this plan to the researcher for confirmation before proceeding.### Step 4 — Write the cleaning functionsGenerate cleaning code that:- Is organized as functions (for `targets` pipeline integration)- Logs N before and after every exclusion step- Documents every transformation with inline comments explaining WHY- References hypotheses, pre-registration, or decision log entries- Uses construct names from the codebook (not generic names)- Creates decision log entries (in `docs/decisions/`) for every subjective choice**R approach:** Write functions in `R/02_clean.R` using tidyverse. Use `psych::alpha()` / `psych::omega()` for reliability. Create composites with `dplyr::rowMeans()` or `psych::scoreItems()`.**Python approach:** Write functions in `python/02_clean.py` using polars. Use `factor_analyzer` or manual computation for reliability. Create composites with `polars` expressions.### Step 5 — Generate CONSORT-style exclusion flowUse [references/templates/consort-flow.md](references/templates/consort-flow.md) as the template. For each exclusion step, record:- Criterion applied- N removed- N remaining- Cumulative removal percentageSave the flow as both a markdown table and a figure.### Step 6 — Write cleaned dataSave to `data/processed/`:- R: `.rds` (native) + `.csv` (interoperable)- Python: `.parquet` (fast, typed) + `.csv` (interoperable)Update the codebook to document any new variables (composites, transformations).### Step 7 — Summary and next stepsPrint:- Starting N → Final N (and percentage retained)- Number of exclusion steps- Number of new variables created- Scale reliabilities for all composites- Where outputs are savedFollow [_shared/next-steps.md](../_shared/next-steps.md) — suggest `/eda` next.## VoiceMeticulous and transparent. You are the person who writes cleaning code so well-documented that Reviewer 2 has nothing to complain about. Every line has a reason. Every exclusion has a count. You show your work.## Argument handlingSame as other skills. Defaults to project root, works with specified paths.

Frequently asked questions