Synthetic Data

Synthetic Data Generation

Generate structurally realistic synthetic datasets for pilot testing, power analysis, and method development.

Modes

Default: From schema. If an experiment design document exists in docs/, auto-select From design. If user provides published statistics, auto-select Calibrated.

When to Use

• Testing analysis code before real data collection
• Power analysis via simulation (complements experiment-design Power mode)
• Method development and debugging estimation pipelines
• Generating pilot data for grant proposals or ethics applications
• Teaching demonstrations with realistic-looking data

When NOT to Use

• Designing the experiment itself --> experiment-design
• Running analysis on real data --> data-analysis
• Auditing identification strategy --> causal-design

Workflow

Step 1: Detect Mode

Detect from context or ask:

Step 2: Interview for Data Structure

Gather the following (adapt questions to mode):

For From design mode, extract most parameters from the design document automatically and confirm with the user.

For Calibrated mode, require the user to provide published summary statistics. Read references/calibration-targets.md for the calibration procedure.

Step 3: Generate Script

Read references/dgp-recipes.md for code patterns matching the requested design.

Read shared/multi-language-conventions.md for language-specific code style.

Generate a self-contained script that:

1. Sets a random seed (document the seed value)
2. Defines the data generating process with clear comments
3. Generates the dataset
4. Adds realistic noise and distributional features
5. Introduces missing data patterns if requested
6. Saves the dataset as CSV (and optionally .rds/.parquet)
7. Prints a summary of the generated data

Step 4: Execute

For large Monte Carlo sweeps (10k+ simulations, multi-condition grids, or long-running bootstrap): run on [HPC cluster] instead of locally. Drop the generation script into hpc/ with submit.sbatch (compute partition) or sweep.sbatch (array over seeds/conditions) — templates at Task Management templates/slurm/, guide at docs/guides/hpc.md. All SLURM templates log git-sha.txt to OUT_DIR so synthetic datasets remain traceable to the DGP code version.

Run the script and verify:

• Dataset has the expected dimensions
• Variable types are correct
• Treatment/control groups are balanced (if applicable)
• Summary statistics are plausible
• No degenerate columns (all zeros, all missing, zero variance)

Step 5: Save Output

Output routing:

NEVER write to data/raw/. The data-sensitivity rule applies -- synthetic data is not raw data and must be clearly separated.

Create data/synthetic/ if it does not exist.

Step 6: Data Dictionary

Produce a markdown data dictionary alongside the dataset:

# Data Dictionary: {name}

Generated: {date}
Script: `code/generate_synthetic_{name}.R`
Seed: {seed}
N: {sample_size}

| Variable | Type | Description | Distribution | Parameters |
|----------|------|-------------|-------------- |------------|
| id | integer | Unique identifier | Sequential | 1 to N |
| treatment | binary | Treatment assignment | Bernoulli | p = 0.5 |
| outcome | continuous | Primary outcome | Normal | mu = 0, sigma = 1 (control) |
| ... | ... | ... | ... | ... |

## Treatment Effects
- True ATE: {value}
- True CATE by subgroup: {if applicable}

## Missing Data
- Mechanism: {MCAR/MAR/MNAR}
- Rate: {percentage} of {variable}

Key Principles

Reproducibility

• Always set random seeds. Document the seed in the script header and the data dictionary.
• Script must be fully self-contained -- running it again with the same seed produces identical output.

Realism

• Match variable types to realistic distributions. Not everything is normal:
  • Income, reaction times --> log-normal or gamma
  • Likert scales --> ordinal with floor/ceiling effects
  • Count data --> Poisson or negative binomial
  • Proportions --> beta
  • Binary outcomes --> Bernoulli with realistic base rates
• Include realistic noise. Treatment effects should be embedded in noisy data, not clean signal.
• Add heterogeneity. Real data has heterogeneous treatment effects -- consider adding subgroup variation.

Missing Data

• If the real data will have missingness, simulate it:
  • MCAR: Random deletion at a fixed rate
  • MAR: Missingness depends on observed variables (e.g., higher income --> less missing)
  • MNAR: Missingness depends on the missing value itself (e.g., depressed people less likely to respond)
• Document the missingness mechanism in the data dictionary.

Calibration (Calibrated Mode)

• Read references/calibration-targets.md for the full procedure.
• Require published summary statistics as input (means, SDs, correlations, effect sizes).
• Validate that the synthetic data matches the targets within tolerance.
• Report discrepancies between target and achieved statistics.

Mode: From Design

Workflow

1. Locate design document -- check docs/experiment-design.md, docs/pre-analysis-plan.md, log/plans/
2. Extract parameters:
   • Treatment arms and assignment mechanism
   • Primary and secondary outcomes (types, expected distributions)
   • Covariates and their roles
   • Sample size and any clustering/stratification
   • Expected effect sizes from the power analysis
3. Confirm with user -- present extracted parameters and ask for adjustments
4. Generate -- follow Steps 3-6 above

Integration with experiment-design

The design document produced by experiment-design Design mode contains everything needed:

• Hypotheses with expected signs --> treatment effect directions
• Conditions and randomization --> treatment assignment DGP
• Outcome measures and scales --> variable types and distributions
• Power analysis results --> sample size and effect size

Mode: From Schema

Workflow

1. Interview -- ask user to describe each variable (or provide a sketch)
2. Infer structure -- detect relationships from the description:
   • "X causes Y" --> include causal pathway
   • "A and B are correlated" --> induce correlation via shared latent factor
   • "C moderates the effect" --> include interaction term
3. Generate -- follow Steps 3-6 above

Common Requests

Mode: Calibrated

Workflow

1. Collect targets -- user provides published summary statistics:
   • Means and standard deviations
   • Correlation matrix (or key correlations)
   • Effect sizes from regression tables
   • Sample size
   • Distribution shapes (skew, kurtosis) if reported
2. Read references/calibration-targets.md for the calibration procedure
3. Generate with matching -- use the Cholesky decomposition or copula method to match the target correlation structure
4. Validate -- compare synthetic descriptives to targets, report match quality
5. Save -- follow Steps 5-6 above, include calibration targets in the data dictionary

Cross-References