Data Deposit Preparation
Prepare an AEA Data Editor compliant replication package for the sewage-house-prices project.
Input: $ARGUMENTS — output directory (defaults to Replication/).
Project-Specific Context
Pipeline Structure
The project has a 6-layer data pipeline in scripts/R/:
01_data_ingestion/ — Raw data collection (EDM archives, APIs)
02_data_cleaning/ — Format standardisation, geocoding, validation
03_data_enrichment/ — Temporal aggregation, rainfall metrics, dry spill identification
04_feature_engineering/ — Spatial matching (house/rental ↔ spill sites)
05_data_integration/ — Merging historical and API EDM data
06_analysis_datasets/ — Final dataset assembly
Analysis scripts: scripts/R/09_analysis/ (6 subdirectories by approach)
Utilities: scripts/R/utils/
Python scripts: scripts/python/ (river network processing)
Docker pipelines: RiverNetworks/, upstream_downstream/
Data Layout
data/raw/ — Original immutable data (EDM, Land Registry, Met Office, shapefiles)
data/processed/ — Intermediate pipeline outputs (parquet)
data/final/ — Analysis-ready datasets
data/cache/ — Postcode geocoding cache
Key Dependencies
- R packages managed via
renv (renv.lock)
- Python environment via
uv in scripts/python/
- PostGIS via Docker for river network analysis
Workflow
Step 1: Inventory
- Read all scripts in
scripts/R/ and parse data file references
- Read
renv.lock for package versions
- Scan
output/tables/ and output/figures/ for output files
- Read the manuscript (
docs/overleaf/_main.tex) for table/figure references
- Check
scripts/python/ for Python dependencies
Step 2: Analyse Dependencies
- Parse script dependencies (which scripts create files that others load)
- Map the execution order (follows the 6-layer pipeline, then analysis scripts)
- Cross-reference the full execution order documented in
ReadMe.md
Step 3: Assemble Package
Create in Replication/ (or specified directory):
README.md — AEA format:
- Data availability statement (which data is public vs restricted)
- Computational requirements (R version, packages, PostGIS, Python)
- Program descriptions (what each script does)
- Replication instructions (step-by-step)
- Expected runtime
master.R — Runs everything in order:
# Master replication script for "Sewage in Our Waters"
# Estimated runtime: [X hours]
source(here::here("scripts", "R", "01_data_ingestion", "script.R"))
# ... through all layers
source(here::here("scripts", "R", "09_analysis", "subdir", "script.R"))
install_packages.R — If renv is not used:
install.packages(c("tidyverse", "fixest", "modelsummary", ...))
DEPOSIT_CHECKLIST.md — Pre-deposit verification
Step 4: Validate
Run the 10 verification checks (equivalent to /audit-replication):
- Script execution order is correct
- All data file references resolve
- All output files are generated
- Package versions documented
- No hardcoded absolute paths
- Data provenance documented
- README completeness (AEA format)
- Output cross-reference (every table/figure traced to a script)
- Restricted data properly flagged
- Master script runs without modification
Step 5: Present Results
- Package contents — All files in
Replication/
- Script order — Numbered sequence with dependency graph
- Data availability — Public vs restricted datasets
- Verification result — X/10 checks passed
- Deposit steps — openICPSR / Zenodo instructions
Principles
- AEA Data Editor standards are the target. README format, versions, data access statements.
- Don't rename scripts without approval. Present ordering first, let the user decide.
- Thorough data provenance. Every dataset documented with source, access date, and restrictions.
- Test before declaring ready. Always validate after assembly.
- Document restricted data clearly. Land Registry and Zoopla data may have access restrictions.
1---2name: data-deposit3description: Prepare a replication package for the sewage-house-prices project. Generates AEA-compliant README, master script, numbered script order, install script, and deposit checklist. Validates the package against 10 verification checks. This skill should be used when asked to "prepare replication", "data deposit", "create replication package", or "package for submission".4argument-hint: "[optional: output directory]"5allowed-tools: ["Read", "Grep", "Glob", "Write", "Edit", "Bash", "Agent"]6metadata:7 source: https://skillmd.com/skills/brycewang-stanford/data-deposit8---910# Data Deposit Preparation1112Prepare an AEA Data Editor compliant replication package for the sewage-house-prices project.1314**Input:** `$ARGUMENTS` — output directory (defaults to `Replication/`).1516---1718## Project-Specific Context1920### Pipeline Structure2122The project has a 6-layer data pipeline in `scripts/R/`:231. `01_data_ingestion/` — Raw data collection (EDM archives, APIs)242. `02_data_cleaning/` — Format standardisation, geocoding, validation253. `03_data_enrichment/` — Temporal aggregation, rainfall metrics, dry spill identification264. `04_feature_engineering/` — Spatial matching (house/rental ↔ spill sites)275. `05_data_integration/` — Merging historical and API EDM data286. `06_analysis_datasets/` — Final dataset assembly2930Analysis scripts: `scripts/R/09_analysis/` (6 subdirectories by approach)31Utilities: `scripts/R/utils/`32Python scripts: `scripts/python/` (river network processing)33Docker pipelines: `RiverNetworks/`, `upstream_downstream/`3435### Data Layout3637```38data/raw/ — Original immutable data (EDM, Land Registry, Met Office, shapefiles)39data/processed/ — Intermediate pipeline outputs (parquet)40data/final/ — Analysis-ready datasets41data/cache/ — Postcode geocoding cache42```4344### Key Dependencies45- R packages managed via `renv` (`renv.lock`)46- Python environment via `uv` in `scripts/python/`47- PostGIS via Docker for river network analysis4849---5051## Workflow5253### Step 1: Inventory54551. Read all scripts in `scripts/R/` and parse data file references562. Read `renv.lock` for package versions573. Scan `output/tables/` and `output/figures/` for output files584. Read the manuscript (`docs/overleaf/_main.tex`) for table/figure references595. Check `scripts/python/` for Python dependencies6061### Step 2: Analyse Dependencies62631. Parse script dependencies (which scripts create files that others load)642. Map the execution order (follows the 6-layer pipeline, then analysis scripts)653. Cross-reference the full execution order documented in `ReadMe.md`6667### Step 3: Assemble Package6869Create in `Replication/` (or specified directory):70711. **README.md** — AEA format:72 - Data availability statement (which data is public vs restricted)73 - Computational requirements (R version, packages, PostGIS, Python)74 - Program descriptions (what each script does)75 - Replication instructions (step-by-step)76 - Expected runtime77782. **master.R** — Runs everything in order:79 ```r80 # Master replication script for "Sewage in Our Waters"81 # Estimated runtime: [X hours]8283 source(here::here("scripts", "R", "01_data_ingestion", "script.R"))84 # ... through all layers85 source(here::here("scripts", "R", "09_analysis", "subdir", "script.R"))86 ```87883. **install_packages.R** — If renv is not used:89 ```r90 install.packages(c("tidyverse", "fixest", "modelsummary", ...))91 ```92934. **DEPOSIT_CHECKLIST.md** — Pre-deposit verification9495### Step 4: Validate9697Run the 10 verification checks (equivalent to `/audit-replication`):981. Script execution order is correct992. All data file references resolve1003. All output files are generated1014. Package versions documented1025. No hardcoded absolute paths1036. Data provenance documented1047. README completeness (AEA format)1058. Output cross-reference (every table/figure traced to a script)1069. Restricted data properly flagged10710. Master script runs without modification108109### Step 5: Present Results1101111. **Package contents** — All files in `Replication/`1122. **Script order** — Numbered sequence with dependency graph1133. **Data availability** — Public vs restricted datasets1144. **Verification result** — X/10 checks passed1155. **Deposit steps** — openICPSR / Zenodo instructions116117---118119## Principles120121- **AEA Data Editor standards are the target.** README format, versions, data access statements.122- **Don't rename scripts without approval.** Present ordering first, let the user decide.123- **Thorough data provenance.** Every dataset documented with source, access date, and restrictions.124- **Test before declaring ready.** Always validate after assembly.125- **Document restricted data clearly.** Land Registry and Zoopla data may have access restrictions.