brycewang-stanford Data Documentation Assistant
Help document datasets systematically for replication packages, co-author handoffs, and future reference.
Dataset Documentation Template
When documenting a dataset, capture:
1. Dataset Overview
- Name: (e.g.,
firm_year_panel.parquet) - Location: (relative path from project root)
- Created by: (script that generates it)
- Last updated:
- Unit of observation: (firm-year, loan-quarter, etc.)
- N observations:
- Time coverage:
2. Source Data
| Source | Access | Raw File | Notes |
|---|---|---|---|
| WRDS Compustat | Subscription | raw/compustat_funda.csv |
Annual fundamentals |
| Revelio Labs | Licensed | raw/revelio_positions.parquet |
Via BU server |
| Hand-collected | Manual | raw/manual_coding.xlsx |
See coding protocol |
3. Key Variables
| Variable | Type | Description | Source | Notes |
|---|---|---|---|---|
gvkey |
str | Compustat firm identifier | Compustat | Primary key |
fyear |
int | Fiscal year | Compustat | |
at |
float | Total assets ($ millions) | Compustat | Winsorized 1/99 |
treated |
int | =1 if treated firm | Constructed | See section 4 |
4. Variable Construction
For constructed/derived variables, document:
Variable: treated
Definition: =1 if firm received first PE investment in year t
Construction:
1. Merge PitchBook deals to Compustat on EIN
2. Keep first deal per firm
3. Flag year of first investment
Script: 2a_construct_treatment.py, lines 45-78
5. Sample Filters
Document all filters applied:
| Filter | Observations Dropped | Remaining |
|---|---|---|
| Raw data | - | 150,000 |
| Drop financials (SIC 6000-6999) | 25,000 | 125,000 |
| Require non-missing assets | 5,000 | 120,000 |
| Require 2+ years in panel | 10,000 | 110,000 |
6. Merge Keys
| Dataset A | Dataset B | Key(s) | Match Rate | Notes |
|---|---|---|---|---|
| Compustat | CRSP | gvkey | 95% | Via CCM link table |
| Compustat | PitchBook | EIN | 72% | Manual cleaning needed |
| Revelio | Compustat | company_name | 68% | Fuzzy match, see script |
Quick Commands
- "document this dataset" - Generate full template for a dataset
- "variable list" - Create variable table only
- "merge documentation" - Focus on merge keys and match rates
- "sample flow" - Generate sample filter table
- "codebook" - Formal codebook format for replication package
Codebook Format (for Replication Packages)
================================================================================
CODEBOOK: firm_year_panel.dta
Generated: 2026-01-15
================================================================================
IDENTIFICATION
gvkey Compustat permanent firm identifier
fyear Fiscal year
OUTCOME VARIABLES
roa Return on assets = ni/at (winsorized 1/99)
investment Capex/lagged assets = capx/L.at
TREATMENT VARIABLES
post =1 for years after treatment
treated =1 for firms ever treated
treat_post Interaction: treated × post (DiD coefficient)
CONTROLS
size Log total assets = ln(at)
leverage Book leverage = (dltt+dlc)/at
mtb Market-to-book = (prcc_f×csho)/ceq
FIXED EFFECTS
ff48 Fama-French 48 industry classification
state State of incorporation
================================================================================
Data Provenance Checklist
Before finalizing a dataset, verify:
- All source files documented with access instructions
- Variable definitions are unambiguous
- Sample filters documented with observation counts
- Merge match rates reported
- Winsorization/trimming documented
- Missing value treatment documented
- Script that creates dataset is identified
- Date of creation recorded