Education Data Source Nccs

NCCS Data Source Reference

National Center for Charitable Statistics (NCCS) Form 990 data for private nonprofit colleges and universities (Portal mirror: IPEDS-matched institutions, 1993-2016). Use when IRS-based financial data — revenue, expenses, assets, endowment details — or governance information (board composition, executive compensation) is needed beyond what IPEDS provides. Portal data ends at 2016; for current filings use full NCCS directly. Public institutions do not file Form 990 and are excluded.

The NCCS (National Center for Charitable Statistics) is the principal U.S. repository for empirical data on the nonprofit sector, derived from IRS Form 990 filings. It provides financial depth, governance detail, and historical coverage for private nonprofit colleges and universities that goes well beyond what IPEDS collects.

CRITICAL: Value Encoding

The Education Data Portal encodes ALL categorical variables as integers, not strings. Variable names are lowercase in Portal data. Always verify codes against codebooks.

Context	fips	mult_ein_flag
Portal integer	6 (California)	1 (Yes)
Original NCCS	"06" or string	N/A

See ./references/variable-definitions.md for complete encoding tables.

What is NCCS?

• Operator: Urban Institute, Center on Nonprofits and Philanthropy
• Coverage: All U.S. tax-exempt nonprofit organizations (~3.8M in BMF)
• Primary data source: IRS Form 990 tax filings
• Frequency: Annual (with filing lag)
• Available years: 1989-present (Core Series); 2012-present (Efile); 1993-2016 (Portal)
• Primary identifier: EIN (Employer Identification Number, 9-digit)
• Education relevance: Private nonprofit colleges/universities are 501(c)(3) orgs filing Form 990; NTEE codes B40-B50 cover higher education

Reference File Structure

Decision Trees

What data am I looking for?

Research need?
├─ Financial data for private colleges → ./references/form-990.md
│   ├─ Revenue breakdown → Part VIII (Statement of Revenue)
│   ├─ Expenses by function → Part IX (Statement of Functional Expenses)
│   ├─ Assets and liabilities → Part X (Balance Sheet)
│   └─ Endowment details → Schedule D
├─ Governance/leadership → ./references/form-990.md
│   ├─ Board members → Part VII (Compensation)
│   ├─ Executive compensation → Part VII, Schedule J
│   └─ Policies and procedures → Part VI (Governance)
├─ Organizational characteristics → ./references/nonprofit-data.md
│   ├─ Basic info (name, address, EIN) → BMF
│   ├─ Tax-exempt type → BMF (SUBSECCD)
│   └─ NTEE classification → BMF (NTEECC)
└─ Identify institutions → ./references/ntee-codes.md
    ├─ All higher education → NTEE B40-B50
    ├─ Universities → B40, B41, B42, B43
    └─ Community colleges → B44

Which NCCS dataset should I use?

Dataset selection?
├─ Need universe of all nonprofits?
│   └─ Business Master File (BMF) → ./references/nonprofit-data.md
├─ Need detailed financial variables?
│   ├─ Large organizations (full 990 filers) → Core PC files
│   ├─ All organizations (990 + 990EZ) → Core PZ files
│   └─ Maximum detail (2000+ fields) → Efile database
├─ Need private foundations?
│   └─ Core PF or 990-PF Efile data
└─ Need small grassroots orgs?
    └─ 990-N ePostcard database

How do I connect NCCS to other education data?

Linking NCCS to education data?
├─ Need to match to IPEDS?
│   └─ See ./references/education-relevance.md (EIN-UNITID crosswalk)
├─ Need geographic analysis?
│   └─ Use BMF geocoded addresses + Census crosswalks
├─ Want institutional comparisons?
│   └─ Filter by NTEE codes B40-B50 for higher ed
└─ Analyzing trends over time?
    └─ Use Core data panel (1989-present)

Quick Reference: NCCS Datasets and Variables

Available Datasets

Portal mirror (via fetch_from_mirrors()):

Full NCCS (direct download, outside Portal):

Scope note: The Portal mirror dataset contains NCCS Form 990 data matched to IPEDS institutions only (~2,600 institutions). It does NOT include the full NCCS universe of ~3.8M nonprofits. For non-education nonprofits, NTEE-based filtering, or BMF/Core/Efile data, download directly from NCCS.

Key Identifiers

Portal dataset:

Full NCCS (direct download, outside Portal):

Education NTEE Codes

Portal Variable Name Mapping (Selected)

The Portal dataset has 161 columns. Key mappings from original NCCS/990 names to Portal lowercase names:

Note: The full 161 columns include detailed revenue breakdowns (Part VIII), expense breakdowns (Part IX), and balance sheet items (Part X). Consult the codebook for the complete list. Use get_codebook_url("nccs/codebook_colleges_nccs_form_990") from fetch-patterns.md to download it.

Missing Data Codes

Important: In Portal data, -1/-2/-3 are integer values. However, empirically these codes are rare in the NCCS Portal dataset — most missing data appears as null rather than negative codes. Only a handful of financial columns (e.g., sale_sec_gross_net, changes_net_assets_other) contain any -1/-2/-3 values. A 0 means the organization reported zero; null means the organization did not report. These are distinct conditions.

Data Access

Datasets for NCCS are available via the mirror system. See datasets-reference.md for canonical paths, mirrors.yaml for mirror configuration, and fetch-patterns.md for fetch code patterns.

Codebooks are .xls files co-located with data in all mirrors. Use get_codebook_url() from fetch-patterns.md to construct download URLs.

Truth Hierarchy: When interpreting variable values, apply this priority:

1. Actual data file (what you observe in the parquet/CSV) — this IS the truth
2. Live codebook (.xls in mirror) — authoritative documentation, may lag
3. This skill documentation — convenient summary, may drift from codebook

If this documentation contradicts the codebook, trust the codebook. If the codebook contradicts observed data, trust the data and investigate.

Filtering

The Portal dataset is pre-filtered to higher education institutions matched to IPEDS UNITIDs. NTEE codes are NOT included in the Portal data — for NTEE-based filtering, use the full NCCS BMF directly.

import polars as pl

# Filter by year
df_2015 = df.filter(pl.col("year") == 2015)

# Filter by state (integer FIPS codes)
df_ca = df.filter(pl.col("fips") == 6)  # California

# Filter out null values from financial columns
df_valid = df.filter(
    (pl.col("revenue_total").is_not_null()) &
    (pl.col("revenue_total") >= 0)  # Excludes rare -1/-2/-3 codes
)

# Replace negative codes with null for analysis (precautionary)
df = df.with_columns(
    pl.when(pl.col("revenue_total") < 0)
    .then(None)
    .otherwise(pl.col("revenue_total"))
    .alias("revenue_total_clean")
)

Common Pitfalls

Key Differences: NCCS vs. IPEDS

Exploration Workflow

Using Portal Data (Recommended for Education Research)

1. Fetch data via fetch_from_mirrors("nccs/colleges_nccs_all")

   • Data is pre-filtered to higher education institutions matched to IPEDS
   • Already includes unitid for IPEDS joining
   • 161 financial/organizational variables, 1993-2016

2. Filter and clean

   • Filter by year, state (FIPS), or institution
   • Handle nulls (primary missing data form in Portal)
   • Check for rare negative codes (-1/-2/-3) in financial columns

3. Link to IPEDS using unitid and year columns

   • Join with IPEDS directory, enrollment, finance, etc.

4. Analyze — See variable definitions for meaning and limitations

Using Full NCCS (for Non-Education or Pre-Portal Research)

1. Identify target organizations

   • Filter BMF by NTEE codes (B40-B50 for higher ed)
   • Verify 501(c)(3) status (SUBSECCD = 03)
   • Note EINs for organizations of interest

2. Select appropriate dataset

   • Core PZ for broad coverage
   • Core PC for detailed financials (larger orgs)
   • Efile for maximum detail (governance, compensation)

3. Extract and clean data

   • Download relevant years from NCCS directly
   • Merge with BMF for organizational attributes
   • Handle missing data codes

4. Analyze — See variable definitions for meaning and limitations

Related Data Sources

Topic Index