# Feature Engineering

> Use when selecting, transforming, or creating features for ML models

- Skill: `kienbui1995/feature-engineering` (Agent Skill)
- Install (CLI): `npx skillmds@latest add kienbui1995/feature-engineering`
- Raw SKILL.md: https://api.skillmd.com/api/skills/kienbui1995/feature-engineering/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: kienbui1995 (https://skillmd.com/u/kienbui1995)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/kienbui1995/feature-engineering

---


# Feature Engineering

## When to Use
When preparing data for model training — selecting which signals to use, how to encode them, and what new features to derive.

## Core Jobs

### 1. Understand the Data
Before engineering:
- Distribution of each feature (histogram, describe())
- Missing value rates
- Correlation with target (mutual information, Pearson)
- Correlation between features (avoid multicollinearity for linear models)

### 2. Handle Raw Features
**Numeric**:
- Scale: StandardScaler (normal dist), MinMaxScaler (bounded), RobustScaler (outliers)
- Transform: log(x+1) for skewed distributions

**Categorical**:
- Low cardinality (< 15 values): one-hot encoding
- High cardinality (> 50 values): target encoding, embedding
- Ordinal: label encode with meaningful order

**Text**:
- TF-IDF for sparse models
- Embeddings (sentence-transformers) for semantic similarity

**Temporal**:
- Extract: hour, day of week, month, is_weekend
- Cyclical: sin/cos encoding for hour/day
- Lag features: value at T-1, T-7, T-30

### 3. Create Derived Features
- Ratios: clicks/impressions, revenue/user
- Differences: current_price - avg_price
- Aggregations: user's avg purchase in last 30 days
- Interaction features: feature_A × feature_B (for linear models)

### 4. Feature Selection
- Remove zero-variance features
- Remove features with > 50% missing
- Use permutation importance or SHAP for post-hoc selection
- Regularization (L1/Lasso) for automatic selection

## Key Outputs
- Feature engineering pipeline (sklearn Pipeline or similar)
- Feature importance report
- Missing value and encoding strategy doc

## Anti-Patterns
- Feature leakage (using future data to predict past)
- Fitting scalers on the full dataset (fit on train only)
- Creating 200 features and not selecting — leads to overfitting
- Forgetting to apply same transformations at inference time