# Customer Inactivity

> Build leakage-safe features for banking customer inactivity and dormancy models. Use this skill whenever the goal is to predict whether a customer will become inactive, dormant, or disengaged over the next 30/60/90 days — for example, no card usage, no digital engagement, no payments, no spend, or no product activity. Covers lifecycle, recency, frequency, monetary behavior, spend trends, digital engagement, servicing and risk signals, and explicit first-vs-last-period comparisons. This is the banking adaptation of the inactivity-feature workflow, tailored for credit cards, deposits, loan products, and customer behavior monitoring.

- Skill: `nageshwarreddyl/customer-inactivity` (Agent Skill, multi-file: 4 files)
- Install (CLI): `npx skillmds@latest add nageshwarreddyl/customer-inactivity`
- Raw SKILL.md: https://api.skillmd.com/api/skills/nageshwarreddyl/customer-inactivity/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: NageshwarReddyL (https://skillmd.com/u/nageshwarreddyl)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/nageshwarreddyl/customer-inactivity

---


# Customer Inactivity / Dormancy Prediction for Banking

A prescriptive workflow for building feature matrices for banking customer inactivity models. Each row is `(customer_id, cutoff_date)` and **every feature must be computable from data observable before `cutoff_date`**. The objective is to predict whether a customer will become dormant or disengaged within a defined future window.

## Step 0 — Pin down the problem

Before writing code, confirm the following:

1. **Inactivity definition.** Define inactivity as a customer showing no meaningful activity within the next `N` days. In banking, examples include:
   - no card transactions in the next 30/60/90 days
   - no digital login/app usage in the next 30 days
   - no payment / deposit / spend activity in the next 60 days
   - no engagement with the account or product family for the next 90 days
2. **Cutoff date (`cutoff_date`).** The as-of date used for prediction. Features can only use data with `timestamp <= cutoff_date`.
3. **Prediction horizon.** Common windows are 30, 60, and 90 days. Longer horizons help label stability; shorter horizons are more operationally useful.
4. **Target semantics.** Decide whether inactivity is measured by card transactions only, total product activity, digital engagement, or a combination.

If the user cannot define these, stop and clarify before engineering features.

### Panel and label construction

Create a panel of `(customer_id, cutoff_date)` rows for every customer with at least some recent activity before the cutoff.

**Recommended cutoff pattern:**
- Daily or weekly snapshots for a period spanning at least 6–12 months
- Monthly snapshots for production stability

**Right-censoring:** exclude rows where `cutoff_date + horizon > data_end_date`.

```python
data_end_date = transactions["txn_ts"].max()
panel = panel[
    panel["cutoff_date"] + pd.Timedelta(days=horizon) <= data_end_date
]
```

**Label logic:** `y = 1` if the customer has no qualifying activity in `(cutoff_date, cutoff_date + horizon]`.

```python
horizon_td = pd.Timedelta(days=horizon)
merged = panel[["customer_id", "cutoff_date"]].merge(
    transactions[["customer_id", "txn_ts"]], on="customer_id", how="left"
)
active_in_window = (
    (merged["txn_ts"] > merged["cutoff_date"]) &
    (merged["txn_ts"] <= merged["cutoff_date"] + horizon_td)
)
active_pairs = (
    merged[active_in_window]
    .drop_duplicates(subset=["customer_id", "cutoff_date"])
    [["customer_id", "cutoff_date"]]
    .assign(had_activity=True)
)
panel = panel.merge(active_pairs, on=["customer_id", "cutoff_date"], how="left")
panel["y"] = (~panel["had_activity"].fillna(False)).astype(int)
```

**Label definition must be explicit:** if inactivity means “no card usage,” use transaction activity only. If it means “no banking activity at all,” combine card, payment, login, and balance events.

## Step 1 — Map the banking schema

The banking version assumes the following canonical tables:

| Table | Grain | Key columns |
|---|---|---|
| `customers` | one row per customer | `customer_id`, `signup_ts`, `dob`, `acquisition_channel`, `segment`, `kyc_level`, `account_status` |
| `cards` | one row per card/account | `card_id`, `customer_id`, `card_type`, `issue_ts`, `status`, `credit_limit`, `product_tier` |
| `transactions` | one row per transaction | `txn_id`, `customer_id`, `card_id`, `txn_ts`, `amount`, `merchant_category`, `merchant_id`, `channel`, `auth_result`, `fraud_flag`, `is_refund`, `is_adjustment` |
| `payments` | one row per payment | `payment_id`, `customer_id`, `card_id`, `payment_ts`, `amount`, `method`, `status` |
| `balances` | one row per account snapshot | `customer_id`, `date`, `current_balance`, `statement_balance`, `available_credit`, `credit_limit` |
| `digital_sessions` | one row per app/portal/login session | `customer_id`, `session_ts`, `session_id`, `channel`, `session_duration`, `event_type` |
| `applications` | one row per application | `application_id`, `customer_id`, `product`, `app_ts`, `app_status`, `decision_ts` |
| `support_tickets` | one row per customer service event | `ticket_id`, `customer_id`, `created_ts`, `resolved_ts`, `issue_type`, `severity`, `sentiment` |
| `bureaus` | one row per bureau pull | `customer_id`, `pull_ts`, `score`, `delinquency_counts`, `open_accounts`, `total_credit_limit` |
| `campaign_events` | marketing or cross-sell touchpoints | `customer_id`, `campaign_id`, `channel`, `event_ts`, `click_flag`, `presentment_id` |

Map the real warehouse tables to these canonical fields before building features. Naming drift is a common source of silent model failure.

## Step 2 — Leakage hygiene (non-negotiable)

Every feature must be answerable from the perspective of the customer at `cutoff_date` without future information.

Rules:

1. Filter all event tables by `timestamp <= cutoff_date` before aggregating.
2. For mutable account state, reconstruct as-of values at `cutoff_date` instead of reading the current table.
3. For late-settling events, use `settlement_ts` or `close_ts` with a `settlement_buffer_days` where needed.
4. Exclude bureau pulls or decision data that occur after the cutoff.
5. Exclude marketing or digital events that happen after the activity label starts.

Examples of leakage traps in banking:
- using post-application approval/decision variables as features
- using future payment resolution or dispute outcomes before settlement
- using future bureau data or account updates introduced after the model snapshot
- using future card presentment events after the inactivity label begins

## Step 3 — Window strategy

Choose rolling windows based on the target horizon.

Use this default set unless the use case strongly suggests otherwise:
- short: 7, 14, 30 days
- medium: 60, 90 days
- long: 180, 365 days

For activity and churn modeling, a common pattern is:
- immediate recency: 7/14/30d
- baseline behavior: 60/90/180d
- long-term trend: 365d

Record the window list in `feature_spec.yaml`.

## Step 4 — Feature families

Build features in independent families and validate leakage after each family.

### 4.1 Lifecycle and customer context
- `tenure_days = cutoff_date - signup_ts`
- `days_since_first_txn`, `days_since_first_payment`, `days_since_first_card_issue`
- `is_new_customer_30d`, `is_new_customer_90d`
- `account_age_bucket` (new / active / mature / dormant)
- `customer_segment`, `acquisition_channel`, `kyc_level`
- `has_credit_card`, `has_debit_card`, `has_savings`, `has_loan`
- `n_active_products`, `product_depth_score`

### 4.2 Recency features
These are often the strongest inactivity signals.

- `days_since_last_txn`
- `days_since_last_payment`
- `days_since_last_login`
- `days_since_last_app_session`
- `days_since_last_email_open`
- `days_since_last_card_use`
- `days_since_last_support_contact`
- `active_in_last_7d`, `active_in_last_30d`, `active_in_last_90d`
- `days_since_last_merchant_category_usage` (top category)
- `days_since_last_product_usage` (top product family)
- `recency_decay_score = sum(exp(-days_ago / k))`

### 4.3 Frequency and cadence features
- `txn_count_{7,14,30,60,90,180}d`
- `card_active_days_{30,90}d`
- `payment_count_{30,90}d`
- `login_count_{30,90}d`
- `app_session_count_{30,90}d`
- `avg_days_between_txns_90d`
- `median_days_between_txns_90d`
- `std_days_between_txns_90d`
- `is_overdue = days_since_last_txn > 1.5 * median_days_between_txns`
- `orders_per_week` analog in banking = `txns_per_week_90d`

### 4.4 Monetary / spend features
- `total_spend_{30,90,180}d`
- `avg_txn_amount_{30,90}d`
- `median_txn_amount_{90d}`
- `max_txn_amount_{90d}`
- `std_txn_amount_{90d}`
- `spend_per_active_day_{30,90}d`
- `aov_last_30d / aov_lifetime`
- `spend_decline_pct_30d`, `spend_decline_pct_90d`
- `payment_amount_total_{30,90}d`
- `cashflow_share_by_category`

### 4.5 Balance and utilization features
- `current_balance_total`, `statement_balance_total`, `available_credit_total`
- `credit_utilization_current`, `credit_utilization_30d`, `credit_utilization_90d`
- `utilization_trend_30d_vs_90d`
- `max_utilization_last_90d`
- `balance_drop_pct_30d`, `recent_balance_decline`
- `revolving_balance_share_{30,90}d`
- `credit_limit_growth_{90d}`

### 4.6 Payments and repayment behavior
- `last_payment_days_ago`
- `on_time_payment_rate_{30,90,365}d`
- `missed_payment_count_{30,90,365}d`
- `payment_lag_mean_days`, `payment_lag_p90_days`
- `autopay_enrolled_flag`, `autopay_usage_rate_{90d}`
- `payment_method_share` by ACH / branch / transfer / cheque
- `payment_to_balance_ratio_{30d}`

### 4.7 Delinquency, bureau, and default-risk signals
- `past_due_bucket_30d`, `_60d`, `_90d`
- `max_past_due_days_last_365d`
- `bureau_score_asof`, `bureau_active_derogatory_count`
- `bureau_total_balance`, `bureau_total_credit_limit`
- `recent_score_drop` and `bureau_score_trend_90d`
- `collections_contact_count_{90d}`
- `collections_amount_sum_{365d}`

### 4.8 Fraud, chargeback, and servicing signals
- `chargeback_count_{90d,365d}`
- `chargeback_rate_{90d}`
- `fraud_flag_recent`
- `auth_failure_rate_{30d}`
- `suspicious_merchant_share_{90d}`
- `support_ticket_count_{30,90}d`
- `customer_complaint_rate_{180d}`
- `avg_support_sentiment_{90d}`

### 4.9 Digital engagement features
- `digital_session_count_{7,30,90}d`
- `app_login_count_{30,90}d`
- `avg_session_duration_{30,90}d`
- `portal_usage_rate_{30d}`
- `digital_engagement_slope_60d`
- `mobile_vs_web_share`
- `email_open_rate_{30,90}d`
- `email_click_rate_{30,90}d`
- `last_app_login_days_ago`

### 4.10 Trend and comparison features (critical)
These are essential for actual dormancy detection.

- `txns_first_6m`, `txns_last_6m`, `txns_last6m_vs_first6m_ratio`
- `spend_first_6m`, `spend_last_6m`, `spend_last6m_vs_first6m_ratio`
- `payments_first_6m`, `payments_last_6m`, `payments_change_pct_6m`
- `txns_last_30d_vs_prev_30d`
- `spend_last_30d_vs_prev_30d`
- `digital_sessions_last_30d_vs_prev_30d`
- `txn_slope_30d`, `txn_slope_90d`
- `spend_slope_60d`, `spend_slope_90d`
- `trend_direction_90d` (up / flat / down)
- `recent_activity_drop_percent_30d`
- `usage_change_6m` and `engagement_change_6m`

These answer the important business question: is the customer slowly disengaging or already dormant?

### 4.11 Product and merchant affinity features
- `top_merchant_category_share_{90d}`
- `top_product_family_share_{90d}`
- `merchant_category_entropy_{90d}`
- `spend_share_grocery_{90d}`, `spend_share_travel_{90d}`
- `preferred_channel` (digital / branch / mobile / ATM)
- `product_affinity_score_card`
- `merchant_repeat_rate_{90d}`
- `new_merchant_share_{30d}`

### 4.12 Interaction features
- `utilization × bureau_score_bucket`
- `recent_spend_drop × recent_login_drop`
- `payment_lag × delinquency_flag`
- `support_issue_count × no_txn_30d`
- `recent_chargeback_count × product_affinity_score`
- `digital_engagement × transaction_count`

### 4.13 Geo and location features
- `n_distinct_cities_last_90d`
- `top_geo_share_90d`
- `geo_region_diversity`
- `travel_spend_share`
- `address_change_flag_last_180d`

### 4.14 Seasonality and calendar features
- `month_of_year_signal`
- `holiday_spend_index`
- `weekday_vs_weekend_spend_share`
- `payday_proximity_flag`
- `statement_cycle_position`

## Step 5 — Feature quality, missingness, and denominators

- Always keep denominators with ratios: `total_txn_count`, `total_spend`, `sessions_count`, `payments_count`
- Replace zero denominators gracefully with `0` or a missing flag where appropriate
- For `days_since_*` features, use a sentinel such as `9999` and a companion `has_*` flag
- Record missingness flags for critical features where missingness itself signals risk

## Step 6 — Leakage checklist for banking models

- No future transaction or payment behavior after cutoff
- No future account status updates
- No future bureau pull or score after cutoff
- No post-outcome digital nudges/marketing events
- No late-settling refunds, chargebacks, or dispute resolution that occur after the observation window

## Step 7 — Validation and selection

- Check distribution drift across monthly cutoffs
- Evaluate target-rate stability by time period
- Remove constant or near-constant features
- Measure feature importance and SHAP values
- Remove highly collinear features with correlation pruning
- Keep business-relevant and interpretable features, not just a long raw list

## Step 8 — Tests and reproducibility

- Unit tests for each feature function
- Check zero-denominator handling
- Validate off-by-one logic with synthetic fixtures
- Use a feature registry to record each feature name, source, window, type, and owner

## Step 9 — Production monitoring

- Monitor PSI / drift for top features
- Alert on sudden null rate increases, stale feed delays, or distribution shifts
- Shadow-run new features and compare with baseline before rollout
- Track uplift in retention or dormancy recall vs production scorecards

## Step 10 — Banking-specific summary

This skill is designed for customer inactivity and dormancy modeling in financial services, especially for:
- card inactivity / dormancy risk
- debit and banking account disengagement
- digital engagement attrition
- product-level inactivity and cross-sell risk
- customer retention and activation monitoring

The same core pattern generalizes across domains:
- lifecycle and tenure features
- recency-frequency-monetary features
- trend and period-comparison features
- product affinity and channel preference
- digital engagement and risk signals
- leakage-safe time windows and as-of logic

## Deliverables

- A feature-generation pipeline using a `(customer_id, cutoff_date)` panel
- A leakage-safe pipeline with as-of filtering
- A tested feature registry covering each feature family
- Reusable templates for other banking and non-banking domains

This banking edition keeps the original modeling discipline of the inactivity workflow but rewrites the business context to customer dormancy in financial products.
- `n_positive_ratings_last_90d` — count of `rating >= 4`.
- `rating_trend_30d_vs_lifetime = avg_rating_last_30d - avg_rating_lifetime`. Falling ratings predict drop-off.
- `n_orders_after_low_rating_last_180d` — recovery behavior. Reordering after a 1–2 star rating is a forgiveness signal.

### 3h. Support & complaint signals

- `n_tickets_last_30d`, `n_tickets_last_90d`, `n_tickets_last_180d`
- `n_high_severity_tickets_last_90d`
- `days_since_last_ticket` (also in 3b — keep in whichever family makes sense for your team)
- `avg_resolution_hours_last_90d` — long resolution times correlate with frustration.
- `support_resolution_rate_last_90d = n_resolved_tickets / n_tickets` over tickets created in window.
- `n_tickets_open_at_cutoff` — created before cutoff, `resolved_ts` after cutoff or still NULL. See leakage notes.
- `tickets_per_order_last_90d` — frustration normalized by usage volume.
- If `sentiment_score` is available: `min_sentiment_last_90d`, `share_negative_sentiment_last_90d`.
- Distribution across `issue_type` last 90d — delivery vs payment vs app issues drive different inactivity dynamics.

### 3i. Promotions & price-sensitivity features

The food-delivery analog of "plan & pricing" — does the customer engage organically or only when discounted?

- `voucher_usage_rate_last_90d` — share of orders with `voucher_used = True`.
- `n_voucherless_orders_last_30d` — organic engagement count.
- `share_orders_with_promotion_last_90d` — orders with `promotion_code` non-null.
- `n_distinct_promotion_codes_used_last_180d` — deal-hunter signal.
- `voucher_dependency_trend` — voucher usage rate last 30d vs prev 60d. Rising reliance on promos is a strong inactivity signal: the customer no longer values the product at full price.
- `avg_order_value_with_voucher_vs_without_last_90d` — price-sensitivity gap.
- `first_order_used_voucher` (binary) — was the activation order discounted? Customers who only ever ordered with a discount go inactive faster.
- `orders_within_7d_of_promo_email_last_90d` — marketing responsiveness. Join `emails` (where `email_type = 'promo'`) to subsequent `orders` within a 7-day attribution window.
- *Note*: the schema has no `discount_amount` column, so "average discount per order" can't be computed exactly. The voucher flag and promotion code identity are the only available proxies.

### 3j. Cohort & acquisition

- `acquisition_channel` (categorical)
- `signup_cohort_month` — use cautiously, can leak temporal trends; prefer relative cohort features below.
- Cohort-relative: `tenure_days_vs_cohort_median`, `orders_30d_vs_cohort_median`, `order_value_vs_cohort_median`.
- `signup_promo_flag` if derivable (signed up under a discount/trial).

### 3k. Marketing engagement features

Customers usually stop opening emails and pushes *before* they stop ordering — these are leading indicators.

- `n_emails_received_last_30d`, `n_emails_opened_last_30d`, `n_emails_clicked_last_30d` — raw counts.
- `email_open_rate_last_90d = n_emails_opened / n_emails_received` (NaN if no emails sent — see Step 5).
- `email_click_rate_last_90d`
- `n_pushes_received_last_30d`, `n_pushes_opened_last_30d`
- `push_open_rate_last_90d`
- Trend: `email_open_rate_last_30d / email_open_rate_prev_60d` — declining engagement.
- Distribution of `email_type` opened last 90d — which content resonates (promo vs transactional vs reactivation).

### 3l. Restaurant / merchant context

These features describe the *quality* of the customer's order history. **Be careful** — many `restaurants` columns are mutable; see Step 2.

- `avg_restaurant_rating_ordered_from_last_90d` — weighted by order count. **Caveat**: `restaurants.avg_rating` is dynamic. If you can't snapshot it at `cutoff_date`, drop this feature or use a historical version.
- `share_orders_from_brg_restaurants_last_90d` — same caveat for `is_brg`.
- `distinct_cities_ordered_in_last_180d` — travel / multi-location signal.
- If you have both `orders.customer_address` and `restaurants.restaurant_address`: approximate `avg_delivery_distance_last_90d`. Use a static geocoder; do not rely on a dynamic distance service.

## Step 4 — Categorical encoding

### Decision rules by cardinality

| Cardinality | Approach |
|---|---|
| 2 (binary) | Cast to `int` (0/1). No encoder. |
| 3–10 | One-hot. Tree models handle them fine; linear models need them. |
| 10–~50 | **Frequency encoding** (`value → count(value) / N`). Cheap, leakage-free, often as strong as target encoding. Implement as `series.value_counts(normalize=True)` on the train set; save the dict, apply via `.map(dict).fillna(0)` at inference. |
| 50+ | **Target encoding with out-of-fold CV + smoothing** (`target_encode_oof`). Use `target_encode_time_aware` if the training set has multiple cutoff dates. |
| ≥10k | **Hash encoding** (`hash_encode`) — fixed bucket count, no per-value artifact. |
| Free text (`subject_line`, `content_summary`) | Out of scope for tabular FE; treat with a separate NLP pipeline. |

### Per-column recommendations for this schema

| Column | Cardinality | Recommended |
|---|---|---|
| `voucher_used`, `is_refunded`, `open_status`, `click_status` | 2 | Cast to int |
| `severity` (low/med/high/critical) | 4 | Ordinal (low=0…critical=3) — levels are ordered |
| `delivery_status`, `acquisition_channel`, `email_type`, `push_type`, `session_type` | 5–10 | One-hot |
| `issue_type` | 10–20 | Frequency encoding + per-value share columns (`categorical_distribution_shares`) |
| `cuisine` | 30–60 | Frequency encoding (also: entropy, top-cuisine-share via helpers) |
| `marketing_channel` | 10–30 | Frequency encoding |
| `city` | 50–500 | Frequency encoding; target encoding on very large platforms |
| `promotion_code` | 100s–1000s | Target encoding (OOF, smoothed). Hash if >10k |
| `restaurant_id` | 10k+ | Target encoding (smoothing 50+) OR hash to 256–1024 buckets |
| `postcode` from `customer_address` | 1k+ | Target encoding OR hash |

When in doubt, frequency encoding is the safe default — never leaks, surprising signal.

### Smoothing strength for target encoding

`target_encode_oof` and `target_encode_time_aware` take a `smoothing` parameter. It pulls rare categories toward the global mean so a restaurant seen once with `y=1` doesn't get encoding `1.0`.

| Smoothing | Behavior |
|---|---|
| 0 | No shrinkage. Rare categories overfit. Avoid. |
| 1–5 | Light. Use when most categories have ≥20 rows. |
| **10** | **Default.** Good balance for typical food-delivery cardinality. |
| 50–100 | Heavy. Use for `restaurant_id` and other long-tailed columns where many values have <5 rows. |

### The biggest target-encoding bug

Computing `mean(y | x)` on the full training set and then training on the same set: every row sees its own label in the encoding and the model memorizes the encoding. `target_encode_oof` does the right thing (out-of-fold).

**Multi-cutoff training sets** add a second leak: OOF folds don't respect time, so a Jan row gets encoded using labels from July rows — and the July labels reflect events from Jan–Oct, leaking the future into the past. `target_encode_time_aware` fixes this by using only labels at strictly earlier cutoffs for each row. See leakage.md Trap 12.

### CatBoost / LightGBM native handling

Modern GBMs encode low-to-mid cardinality categoricals natively — pass `cuisine`, `marketing_channel`, `issue_type` directly. For high-cardinality columns (`restaurant_id`, `postcode`) and **any multi-cutoff training set**, use manual target/hash encoding — native encoders don't respect `cutoff_date` time semantics.

## Step 5 — Missing-value semantics

The right imputation depends on *why* a value is missing. Make the distinction explicit:

| Pattern | Meaning | Treatment |
|---|---|---|
| `n_orders_30d = NaN` | No orders found in window | Replace with `0`. Missing = no activity. |
| `days_since_last_order = NaN` | Customer has never ordered (`has_ever_ordered = False`) | Replace with a large sentinel (e.g., `9999`) **and** keep the `has_ever_ordered` flag. The sentinel + flag combination lets tree models distinguish "never ordered" from "ordered long ago" cleanly. |
| `refund_rate_last_90d = NaN` | No orders in window (denominator is 0) | Replace with `0` **and** carry the denominator `n_orders_last_90d` as a separate feature so the model can weight the rate appropriately. |
| `email_open_rate_last_90d = NaN` | No emails sent (denominator is 0) | Same treatment as above — `0` plus the denominator. |
| `sentiment_score = NaN` | Ticket exists but sentiment service didn't score it | True unknown. Median-impute **and** add a `sentiment_missing` flag. |
| `rating = NaN` on an order | Customer didn't rate that order | True unknown. Aggregations like `avg_rating_given_last_90d` should ignore NaNs at aggregation time; never impute zero (zero ≠ "no rating"). |
| `address = NaN` | Customer never entered one | Flag as `address_missing`; do not invent a value. |

Always add a `_missing` indicator column for true-unknown imputations. Tree models exploit them; linear models still benefit.

## Step 6 — Validate and filter the feature matrix

### 6a. Validation assertions

Before handing off to modeling, assert:

1. **No future timestamps.** For each row, the max timestamp of any event used to compute features is `<= cutoff_date`. Use `scripts/inactivity_features.py:assert_no_future_leakage`.
2. **Inactivity rate is sane.** Compute `mean(y)` and compare to the user's expected inactivity rate from product analytics. Off by an order of magnitude? Either the inactivity definition or the join is wrong — stop and debug.
3. **Feature distributions stable across cutoffs.** Food delivery has strong weekly and monthly seasonality (weekends, paydays, holidays). If the training set has multiple cutoff dates, plot key feature means by cutoff. Big jumps either reflect real seasonality (document and consider seasonality-adjusted features) or a data pipeline change (find and fix it).
4. **No constant columns.** Drop them.
5. **No columns perfectly correlated with the label.** If you find one, it is almost certainly leakage. Investigate before celebrating.
6. **Settlement-buffer check for refunds & delivery status.** Refunds and final delivery status sometimes settle days after the order. Confirm orders are fully settled as-of `cutoff_date` — e.g., only count orders with `order_ts <= cutoff_date - settlement_buffer_days`. Otherwise late-settling refunds sneak into "pre-cutoff" features.

### 6b. Filter to ~50 features (correlation + mutual info, model-agnostic)

After Step 3 the raw matrix typically has 100–300 columns. Many are redundant by construction — rolling-window twins like `orders_last_7d` and `orders_last_14d` are highly correlated, as are `aov_last_30d` and `aov_last_90d`. A long tail also has weak signal. Filter down to a model-agnostic shortlist (~50) before training: less wasted compute, cleaner feature-importance plots, easier debugging.

**Model-agnostic** means: no XGBoost importance, no SHAP, no LightGBM gain — nothing that requires having trained a model first. The 50 you pick should be defensible regardless of which model you train next.

**Three-step procedure (one call):**

1. Encode categoricals to numeric (use OOF / time-aware encoding from Step 4 — raw target-encoded values leak).
2. Drop near-duplicates: for each pair with `|spearman| > 0.95`, drop the lower-MI one.
3. Rank survivors by mutual information against `y` and keep the top K.

Use the helper from `scripts/inactivity_features.py`:

```python
from inactivity_features import filter_features_correlation_mi, apply_feature_filter

X_train_filtered, filter_info = filter_features_correlation_mi(
    X_train, y_train,
    top_k=50,
    max_corr=0.95,
    corr_method="spearman",     # robust to skewed counts / amounts
    task="classification",      # use "regression" for continuous outcomes
)

# Persist filter_info alongside the model. Apply at val/test/inference:
X_val_filtered = apply_feature_filter(X_val, filter_info)
```

`filter_info` contains the dropped columns, the MI ranking of survivors, the final top-K column list, and the thresholds used — persist it as part of the feature spec (Step 7).

**Sanity checks before locking in the 50:**

- **Family coverage.** The top-50 should include features from a healthy mix of families (3a–3l). All 50 from a single family usually means either leakage in that family (investigate!) or weak engineering in the others.
- **MI floor.** The 50th feature's MI shouldn't be near zero. If it is, the signal ceiling is lower than 50 — pick a smaller K (20–30).
- **Temporal stability.** Re-run the ranking on a held-out future cutoff. If the top-50 lists differ substantially between training and held-out cutoffs, the feature set is unstable — possibly leakage, possibly drift / seasonality. Investigate before shipping.

**Why MI instead of Pearson-with-label?** Mutual information captures non-linear dependence (U-shapes, thresholds, interactions with a small set of values) that tree models exploit. Pearson misses these and ranks them as noise.

**Continuous outcomes** (e.g., revenue rather than binary inactivity): swap `mutual_info_classif` → `mutual_info_regression`.

**Tuning K.** 50 is a reasonable default for inactivity on millions of customers. Drop to 20–30 if labels are scarce or you need a small interpretable model; raise to 100+ if you have abundant data and a GBM backend that handles wide inputs well. Persist the chosen `K`, the correlation threshold (`0.95`), and the surviving column names in the feature spec (Step 7) — these are part of the model contract.

### 6c. Class imbalance and evaluation metrics

Inactivity labels are structurally imbalanced — typically 10–30% positive for a 30-day food-delivery horizon. Standard accuracy is useless: a model predicting all zeros scores 70–90% accuracy while identifying nobody.

**Choose the right metrics:**

| Metric | When to use |
|---|---|
| **AUC-ROC** | Primary metric — threshold-free overall quality. Target > 0.75; > 0.85 is strong; > 0.95 → investigate leakage first. |
| **Precision@K** | When the model drives a fixed-capacity intervention (e.g., reach 1,000 customers/week). "Of the K flagged, what share actually go inactive?" |
| **Recall@K** | When missing inactive customers is costly. "Of all who go inactive, what share did we catch?" |
| **PR-AUC** | Prefer over ROC-AUC when positive rate < 5% — ROC is optimistic at very low positive rates. |
| Accuracy | Never. |

**Handle the imbalance:**

- **Class weights first.** In scikit-learn: `class_weight='balanced'`; in XGBoost/LightGBM: `scale_pos_weight = n_negatives / n_positives`. Reweights the loss without touching the data — do this before anything else.
- **Threshold calibration.** The 0.5 default threshold is wrong for imbalanced data. Sweep thresholds on a held-out validation set and choose based on your intervention capacity — e.g., "we can reach 1,000 customers/week; maximise recall at that volume." Use the precision-recall curve to select the threshold, not the ROC curve.
- **If oversampling (SMOTE):** apply within each cutoff cohort separately. Oversampling across cutoffs leaks — synthetic rows built from a June cutoff using July neighbours import July's label signal into June's training rows.

**Output:** the model produces a ranked score (propensity to go inactive), not a binary flag. Downstream teams select the top-K scores for intervention. Threshold selection is a business decision — intervention capacity and cost of false positives — not a model hyperparameter.

## Step 7 — Version and document the feature spec

For every feature pipeline:

- Save a `feature_spec.yaml` listing every feature, its source table, its rolling window (if any), and its as-of semantics.
- Hash the feature spec and include the hash in the model artifact name. Feature/model version drift is a top-three cause of production model regressions.
- Re-runs of the pipeline with the same `(cutoff_date, feature_spec_hash, raw_data_snapshot)` must produce identical output. If they don't, you have a determinism bug — usually an unordered groupby, a random sample, or a `datetime.now()` call sneaking in.

### Persist encoders alongside the model

Every encoder fit on train must be saved so it can be re-applied at inference. A single `encoders.json` or `.pkl` alongside the model is the standard pattern:

```python
encoders = {
    "acquisition_channel": {
        "kind": "frequency",
        "mapping": {"organic": 0.42, "paid_search": 0.31, "referral": 0.18, ...},
    },
    "restaurant_id": {
        "kind": "target_smooth",
        "global_mean": 0.07,
        "smoothing": 50.0,
        "category_to_value": {"r_001": 0.04, "r_002": 0.11, ...},
    },
    # hash encoder entries (e.g. promotion_code) store only n_buckets + algorithm — no per-value mapping.
}
```

Unseen-category fallbacks at inference:

| Encoder kind | Fallback when category not in mapping |
|---|---|
| `frequency` | `0.0` (never-seen → no signal) |
| `target_smooth` | `global_mean` |
| `time_aware_target` | `global_mean` for the relevant cutoff |
| `hash` | Works for any input — no fallback needed |
| one-hot | All zeros (add an explicit `unknown` column if unseen values are expected) |

Add a unit test: re-apply the encoders to a held-out customer at synthetic `cutoff_date = now`, compare against features produced by the training pipeline at the same cutoff. Mismatches mean train/serve skew.

### Persist the feature filter

The output of Step 6b (`filter_info` dict — `top_features`, `max_corr`, `top_k`, `mi_ranking`) is part of the model contract. Save it next to `encoders.json` and apply at inference via `apply_feature_filter(X, filter_info)`.

## Helpers

Reusable pandas helpers in [scripts/inactivity_features.py](scripts/inactivity_features.py). Use these instead of re-implementing — they handle edge cases (empty windows, ties on cutoff timestamps, single-row groups) that ad-hoc implementations get wrong.

**Joins & windows**: `as_of_join` · `compute_rfm_window` · `compute_rfm_multi_window`

**Lifecycle (3a)**: `time_to_nth_event`

**Recency (3b)**: `days_since_last_event` · `days_since_last_event_from_top_group` · `recency_decay_score`

**Frequency & cadence (3c)**: `distinct_active_days_in_window` · `cadence_stats`

**Trends (3e)**: `build_daily_series` · `compute_usage_slope`

**Concentration & diversity (3f)**: `concentration_hhi` · `shannon_entropy` · `top_group_share` · `categorical_distribution_shares`

**Support tickets (3h)**: `tickets_open_at_cutoff` · `resolution_rate`

**Cross-event attribution (3i)**: `events_within_window_of_anchor`

**Encoders (Step 4)**: `target_encode_oof` · `target_encode_time_aware` · `hash_encode`

**Validation (Step 6a)**: `assert_no_future_leakage`

**Feature filtering (Step 6b)**: `filter_features_correlation_mi` · `apply_feature_filter`

## Reference files

- [references/schema.md](references/schema.md) — canonical schema, column dtypes, mapping template
- [references/leakage.md](references/leakage.md) — catalogue of subtle leakage traps
- [scripts/inactivity_features.py](scripts/inactivity_features.py) — pandas helpers (including all encoders)