Education Data Source Edfacts

EDFacts Data Source Reference

EDFacts — federal K-12 outcome data from State Education Agencies, covering state assessment proficiency rates, ACGR graduation rates, and ESSA accountability indicators at school and district level (assessments 2009-2020, graduation rates 2010-2019). Use when analyzing within-state achievement trends, subgroup proficiency gaps, or adjusted cohort graduation rates. Complements CCD (school characteristics) with outcome data. State assessment scores CANNOT be compared across states; use NAEP for cross-state comparisons.

EDFacts is the U.S. Department of Education's centralized data collection system for pre-K through grade 12 education data from State Education Agencies (SEAs). It provides state assessment proficiency rates, graduation rates, and accountability indicators — the authoritative federal source for state-level K-12 outcome data.

CRITICAL: Value Encoding

The Urban Institute Education Data Portal converts NCES string codes (e.g., ALL, CWD, LEP) to integer codes. Always verify actual data values before filtering — do not rely on documentation labels alone.

Context	Subgroup "All"	English Learner	Sex "Male"
Portal integer	99	1	1
NCES string	ALL	LEP	M

See ./references/variable-definitions.md for complete encoding tables.

What is EDFacts?

• Collector: U.S. Department of Education, via State Education Agencies (SEAs)
• Coverage: All public schools and districts in 50 states + DC
• Content: State assessment proficiency rates, ACGR graduation rates, participation rates, accountability indicators
• Frequency: Annual collection
• Available years: Assessments 2009-10 to present; Graduation rates 2010-11 to present
• Primary identifiers: ncessch (school ID, Int64), leaid (district ID, Int64), fips (state FIPS code, Int64)
• Key limitation: State assessment scores CANNOT be compared across states (different tests, different cut scores)
• Available through: Education Data Portal mirrors

Reference File Structure

Decision Trees

What type of analysis?

What EDFacts data do you need?
├─ Assessment/proficiency data
│   ├─ Within-state trends → Valid analysis
│   ├─ Cross-state comparison → INVALID - use NAEP instead
│   └─ Subgroup gaps → See ./references/subgroup-reporting.md
├─ Graduation rates (ACGR)
│   ├─ Understand methodology → See ./references/graduation-rates.md
│   ├─ Extended rates (5-year, 6-year) → See ./references/graduation-rates.md
│   └─ Subgroup rates → See ./references/subgroup-reporting.md
├─ Understanding variables
│   ├─ Missing/suppressed values → See ./references/variable-definitions.md
│   ├─ Range vs. exact values → See ./references/variable-definitions.md
│   └─ Subgroup codes → See ./references/subgroup-reporting.md
└─ Data quality concerns
    ├─ COVID-19 impacts (2019-20) → See ./references/data-quality.md
    ├─ State reporting changes → See ./references/data-quality.md
    └─ Suppression rates → See ./references/data-quality.md

Is my comparison valid?

What are you comparing?
├─ Same state, different years
│   ├─ Same assessment system? → Valid
│   └─ Different tests? → Break in time series
├─ Schools within same state → Valid
├─ Districts within same state → Valid
├─ Subgroups within same school → Valid (check suppression)
├─ Different states
│   ├─ Proficiency rates → INVALID
│   ├─ Graduation rates (ACGR) → More comparable
│   └─ Use NAEP instead → Valid
└─ National ranking by proficiency → INVALID

Quick Reference: EDFacts Data Elements

Assessment Data

Graduation Data

Key Identifiers

Portal Data Types: All identifiers are Int64 in the Portal parquet files. The NCES source format (zero-padded strings) is shown for reference only. When joining with other Portal datasets, join on the integer columns directly.

Data Levels

Subgroups Reported

Note: Not all subgroup columns are present in every dataset. Grad rates data does NOT have sex, migrant, or military_connected columns.

EDFacts Filter Column Pattern:

• Special population columns (lep, disability, homeless, etc.) use 1 = subgroup, 99 = total
• Race column uses integer codes (1=White, 2=Black, etc.)
• Sex column uses 1 = Male, 2 = Female, 99 = Total (assessments only)

Grade Codes (grade_edfacts)

Race Codes

Empirically verified from 2018 school assessment data. Only these values appear in the race column:

Note: Code 6 (Native Hawaiian/Pacific Islander) is NOT observed in the data. Codes 8 (Nonresident alien), 9 (Unknown), 20 (Other), -1, -2, -3 are also not observed in the race column. These codes may exist in other Portal sources but are absent from EDFacts.

Sex Codes

Disability Codes

Empirically verified from 2018 school assessment and 2019 grad rate data. Only 1 and 99 are observed in the disability column. The expanded codes (0-4) documented in other Portal sources are NOT present in EDFacts datasets.

LEP Codes

Special Population Columns

For homeless, migrant, econ_disadvantaged, foster_care, military_connected:

Missing Data Codes

Always use _midpt variables for analysis when exact values are suppressed.

Data Access

All EDFacts data is fetched via the Education Data Portal mirror system. There is no API access.

Key references:

• mirrors.yaml -- Mirror definitions, URL templates, read strategies
• datasets-reference.md -- Canonical dataset paths (one path works for all mirrors)
• fetch-patterns.md -- fetch_from_mirrors() and fetch_yearly_from_mirrors() patterns

Truth Hierarchy: When interpreting variable values, apply this priority:

1. Actual data file (what you observe in the parquet/CSV) — this IS the truth
2. Live codebook (.xls in mirror) — authoritative documentation, may lag
3. This skill documentation — convenient summary, may drift from codebook

If this documentation contradicts the codebook, trust the codebook. If the codebook contradicts observed data, trust the data and investigate.

Key Datasets

Note: 2019 assessment data is NOT available (at any level) due to COVID testing waivers.

Codebooks

Codebook .xls files are available for both assessment and graduation rate datasets. Use get_codebook_url() from fetch-patterns.md:

# Assessment codebooks:
url = get_codebook_url("edfacts/codebook_schools_edfacts_assessments")
url = get_codebook_url("edfacts/codebook_districts_edfacts_assessments")

# Graduation rate codebooks:
url = get_codebook_url("edfacts/codebook_schools_edfacts_graduation")
url = get_codebook_url("edfacts/codebook_districts_edfacts_graduation")

Codebook naming note: Graduation rate codebooks use _graduation (not _grad_rates), while the data files use _grad_rates. This follows the same pattern as other Portal sources where codebook names differ from data file names. See datasets-reference.md for the authoritative path mapping.

Dataset Column Differences

Assessment and graduation rate datasets have different column sets:

Filtering

# Grade filtering: grade_edfacts uses integer codes
df = df.filter(pl.col("grade_edfacts") == 4)  # Grade 4
df = df.filter(pl.col("grade_edfacts") == 99)  # All grades combined

# Subgroup filtering: special population columns use 1/99 pattern
df_total = df.filter(pl.col("sex") == 99)  # All students (total)
df_econ = df.filter(pl.col("econ_disadvantaged") == 1)  # Economically disadvantaged only

# Race filtering: integer codes
df_black = df.filter(pl.col("race") == 2)  # Black students

Common Pitfalls

Key Policy Context

• AYP (Adequate Yearly Progress): NCLB requirement eliminated by ESSA
• ESSA Accountability: States design own systems with federal guardrails
• N-size: Minimum students required for reporting (varies by state, typically 10-30)

CRITICAL WARNING: Cross-State Comparisons

State assessment proficiency rates CANNOT be compared across states.

A student "proficient" in one state may score "below basic" in another state taking a harder test with higher cut scores. Rankings of states by proficiency rates are meaningless.

Use NAEP (National Assessment of Educational Progress) for valid cross-state comparisons.

Valid vs. Invalid Analysis Examples

Valid Analysis:

# Within-state trend analysis
state_df = df.filter(pl.col("fips") == 6)  # California only
trend = state_df.group_by("year").agg(
    pl.col("read_test_pct_prof_midpt").mean()
)
# Valid: Same state, same test system

INVALID Analysis:

# DO NOT DO THIS - Cross-state comparison
# This comparison is MEANINGLESS
state_comparison = df.group_by("fips").agg(
    pl.col("read_test_pct_prof_midpt").mean()
).sort("read_test_pct_prof_midpt", descending=True)
# INVALID: Different tests, different standards

Related Data Sources

Topic Index