# Ds Feature Engineering > "Creates and transforms features including polynomial features, interactions domain-specific features, and feature transformations" - Skill: `paulpas/ds-feature-engineering` (Agent Skill) - Install (CLI): `npx skillmds@latest add paulpas/ds-feature-engineering` - Raw SKILL.md: https://api.skillmd.com/api/skills/paulpas/ds-feature-engineering/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - License: MIT - Author: paulpas (https://skillmd.com/u/paulpas) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/paulpas/ds-feature-engineering --- # Feature Engineering Comprehensive guide to feature engineering in machine learning and data science workflows. ## When to Use This Skill - Solving real-world feature engineering problems - Building machine learning pipelines with feature engineering - Implementing best practices for feature engineering - Optimizing model performance using feature engineering techniques - Learning industry-standard approaches to feature engineering ## When NOT to Use This Skill - When using pre-built libraries without understanding underlying concepts - For toy problems that don't require feature engineering rigor - When domain expertise in specific problem requires different approach - If your problem doesn't require the complexity this skill provides ## Purpose and Key Concepts Feature Engineering is a critical component of the machine learning workflow. This skill covers: 1. **Theoretical foundations** — Mathematical principles and statistical concepts 2. **Practical implementation** — Working code examples and patterns 3. **Common pitfalls** — Mistakes to avoid and how to recover from them 4. **Best practices** — Industry-standard approaches and optimization techniques ## Core Workflow 1. **Understand the problem** — Clearly define what you're solving for 2. **Select approach** — Choose the right technique for your data and constraints 3. **Implement solution** — Write clean, tested code following best practices 4. **Validate results** — Verify your implementation with tests and validation 5. **Optimize performance** — Improve efficiency and accuracy incrementally ## Implementation Patterns ### Pattern 1: Basic Feature Engineering ```python import pandas as pd import numpy as np from sklearn.preprocessing import PolynomialFeatures, StandardScaler from sklearn.compose import ColumnTransformer from sklearn.pipeline import Pipeline def basic_feature_engineering(df: pd.DataFrame, target_col: str, numeric_cols: list) -> tuple: """ Apply basic feature engineering transformations to a DataFrame. Handles polynomial features, log transforms, and standard scaling. """ if df.empty: raise ValueError("Input DataFrame cannot be empty") # Separate features and target X = df.drop(columns=[target_col]) y = df[target_col] # Identify numeric columns for transformation numeric_features = [col for col in numeric_cols if col in X.columns] # Create transformation pipeline transformer = ColumnTransformer( transformers=[ ('poly', PolynomialFeatures(degree=2, include_bias=False), numeric_features) ('scaler', StandardScaler(), numeric_features) ] remainder='passthrough' ) # Fit and transform data X_transformed = transformer.fit_transform(X) # Create DataFrame with new feature names poly_names = transformer.named_transformers_['poly'].get_feature_names_out(numeric_features) new_df = pd.DataFrame(X_transformed, columns=poly_names, index=X.index) return new_df, transformer, y ``` ### Pattern 2: Production-Ready Feature Engineering ```python import logging import pandas as pd import numpy as np from typing import Any, Dict, List, Optional from sklearn.preprocessing import StandardScaler, OneHotEncoder from sklearn.impute import SimpleImputer from sklearn.compose import ColumnTransformer from sklearn.pipeline import Pipeline import warnings logger = logging.getLogger(__name__) warnings.filterwarnings('ignore') class FeatureEngineering: """Production-grade feature engineering pipeline with validation and logging.""" def __init__(self, numeric_cols: List[str], categorical_cols: List[str], target_col: str, degree: int = 2, handle_missing: bool = True): self.numeric_cols = numeric_cols self.categorical_cols = categorical_cols self.target_col = target_col self.degree = degree self.handle_missing = handle_missing self.pipeline = None self.feature_names = None def _validate_input(self, data: pd.DataFrame) -> None: missing_cols = set(self.numeric_cols + self.categorical_cols + [self.target_col]) - set(data.columns) if missing_cols: raise ValueError(f"Missing required columns: {missing_cols}") if data.isnull().sum().sum() > 0 and not self.handle_missing: logger.warning("Data contains missing values but handle_missing is False") def execute(self, data: pd.DataFrame) -> Dict[str, Any]: """Execute feature engineering on data with full validation and logging.""" self._validate_input(data) logger.info(f"Starting feature engineering on {len(data)} rows") X = data.drop(columns=[self.target_col]) y = data[self.target_col] # Build preprocessing pipeline numeric_transformer = Pipeline(steps=[ ('imputer', SimpleImputer(strategy='median')) if self.handle_missing else None ('scaler', StandardScaler()) ]) categorical_transformer = Pipeline(steps=[ ('imputer', SimpleImputer(strategy='most_frequent')) if self.handle_missing else None ('encoder', OneHotEncoder(handle_unknown='ignore', sparse_output=False)) ]) preprocessor = ColumnTransformer( transformers=[ ('num', numeric_transformer, self.numeric_cols) ('cat', categorical_transformer, self.categorical_cols) ] remainder='drop' ) # Fit and transform X_transformed = preprocessor.fit_transform(X) self.pipeline = preprocessor # Get feature names num_names = preprocessor.named_transformers_['num'].named_steps['scaler'].get_feature_names_out(self.numeric_cols) cat_names = preprocessor.named_transformers_['cat'].named_steps['encoder'].get_feature_names_out(self.categorical_cols) self.feature_names = np.concatenate([num_names, cat_names]) logger.info(f"Feature engineering complete. Output shape: {X_transformed.shape}") return { 'X_transformed': X_transformed 'y': y.values 'feature_names': self.feature_names 'pipeline': self.pipeline 'metadata': {'original_shape': data.shape, 'transformed_shape': X_transformed.shape} } ``` ## Best Practices - ✅ Always validate your implementation on test data - ✅ Document your assumptions and methodology - ✅ Use version control for reproducibility - ✅ Monitor performance metrics in production - ✅ Periodically review and update your approach - ✅ Test with edge cases and outliers - ✅ Log all significant operations for debugging ## Common Pitfalls | Pitfall | Problem | Solution | | --- --- ## Constraints ### MUST DO - Document every engineered feature with its formula, data source, and expected range before adding to model pipeline - Validate new features against the target variable using mutual information or point-biserial correlation - Implement feature transformations in a composable pipeline: fit-only on training data, transform on all splits - Track feature drift over time by monitoring distribution statistics (mean, std, skewness) per batch ### MUST NOT DO - Do not create features that leak the target variable (e.g., using future values or post-event metrics) - Avoid creating too many features without dimensionality reduction — aim for 20-50 high-signal features - Never engineer features using statistics computed over the full dataset including test set — causes data leakage - Do not use custom feature engineering that cannot be reproduced in production — stick to standard transformations ## Live References > Authoritative documentation links for this skill's domain. The model follows markdown links at load time to resolve external references and inline content. - [Feature Engineering — Scikit-learn docs](https://scikit-learn.org/stable/modules/feature_extraction.html) - [Feature Engineering Guide (Towards Data Science)](https://towardsdatascience.com/feature-engineering-for-machine-learning-a-guide-a9f9ea4bb30) - [Featuretools — Automated Feature Engineering](https://docs.featuretools.com/) - [PyCaret Feature Engineering](https://pycaret.org/features/) - [Kaggle Feature Engineering Course](https://www.kaggle.com/learn/feature-engineering)