Enrichment Waterfall

Enrichment Waterfall

Fill fields at the lowest defensible cost per usable record. The design decision that matters is ordering, not vendor selection.

When to use

• Any list needs contact or company data attached
• Match rates are below benchmark
• Enrichment spend is growing faster than the list
• Choosing or replacing a provider

Inputs

• Reads: workspace/data/universe.csv, workspace/data/object-model.md
• Needs from user: which fields are actually required, geography mix, current provider set and pricing, current match rate if known

Workflow

1. Cut the field list

Ask which fields will change a decision. Every field that will not is pure cost.

The usual honest answer for outbound is: verified email, title, seniority, and one firmographic. Everything else is nice to have, and nice to have is billed at the same rate as necessary.

2. Put a free gate in front

This step saves more money than every other step combined, and it costs nothing.

GATE  (free or already-known fields only)
  Employee count in range?          from sourcing data
  Industry in the allowed set?      from sourcing data
  Geography in scope?               from sourcing data
  Not on the suppression list?      local file
  Not already in CRM with data?     local join
        ↓
  Only survivors proceed to paid columns

A well-built gate removes 40 to 70% of rows. If your gate kills under 25%, it is too permissive and you are overspending on rows you will discard anyway.

The gate must be structurally upstream of every paid column, not merely earlier in a list of steps. A paid column that runs on all rows and gets filtered afterward has already billed you.

3. Order the waterfall by cost-to-match ratio

Expected cost per row:

E[cost] = c1 + (1-m1)·c2 + (1-m1)(1-m2)·c3 + ...

Cheapest viable source first. A free source with a 20% match rate at the top removes a fifth of paid volume for nothing.

TIER 1  free or already-owned      → CRM, past exports, known contacts
TIER 2  cheapest paid provider
TIER 3  mid-cost, different underlying data
TIER 4  premium, only for tier-one accounts
        ↓
TIER 5  verification pass on everything found
        ↓
        graceful exit: mark unresolvable, stop

Segment the waterfall by geography. A US mid-market order and a European order should not be identical. Providers have very different regional coverage, and one global order means accepting the weakest region.

4. Measure incremental match per tier

The number that decides whether a tier stays:

Tier N incremental match = records first found at tier N ÷ records reaching tier N

Tier four fires only on records that three sources already missed, which is a population selected for being hard. Incremental match under 5% at a premium price means that tier should be reserved for tier-one accounts or dropped.

5. Verify, always, and count separately

Found is not verified and verified is not deliverable. Run a verification pass and track two numbers separately: match rate and verification pass rate. A provider with a high match rate and a low verification rate is guessing at syntax.

Hard-bounce rate above 5% is a deliverability event. Stop sending and fix the verification step before anything else.

6. Enrich only empty fields

The default is fill-if-empty, never overwrite. Refresh is a separate operation driven by the freshness window in the data dictionary, and it respects the direction-of-truth rule per field.

The overwrite failure mode is quiet and expensive: vendor data replacing hand-verified values, so the best records in the system degrade every cycle.

7. Compute the real number

Cost per usable record = total spend ÷ (verified AND gate-passing records)

Report this. Never report records enriched, which rewards spending more.

Output

• Writes: workspace/data/waterfall-spec.csv, enriched list, and a run log
• Uses: templates/enrichment-waterfall-spec.csv
• Prints: gate kill rate, per-tier match and incremental match, verification pass rate, total spend, cost per usable record

Rules & quality bar

• Free gate upstream of every paid column. Non-negotiable
• Cheapest source first, ordered by cost-to-match ratio
• Geography-specific ordering
• Verification always, counted separately from match
• Enrich only empty fields
• Graceful exit after N misses. No unbounded retry on hard records
• Report cost per usable record, never records enriched
• Test at 5 rows, then 25, then release. Every time

Related skills

• Requires: tam-and-sourcing, data-hygiene upstream
• Hands off to: icp-scoring-engine
• Pairs with: credit-and-cost-control
• See also: docs/cost-model.md for the full arithmetic