# Ds Hyperparameter Tuning > "Optimizes hyperparameters using grid search, random search, Bayesian optimization, and evolutionary methods for model improvement" - Skill: `paulpas/ds-hyperparameter-tuning` (Agent Skill) - Install (CLI): `npx skillmds@latest add paulpas/ds-hyperparameter-tuning` - Raw SKILL.md: https://api.skillmd.com/api/skills/paulpas/ds-hyperparameter-tuning/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - License: MIT - Author: paulpas (https://skillmd.com/u/paulpas) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/paulpas/ds-hyperparameter-tuning --- # Hyperparameter Tuning Comprehensive guide to hyperparameter tuning in machine learning and data science workflows. ## When to Use This Skill - Solving real-world model evaluation & selection problems - Building machine learning pipelines with hyperparameter tuning - Implementing best practices for hyperparameter tuning - Optimizing model performance using hyperparameter tuning techniques - Learning industry-standard approaches to hyperparameter tuning ## When NOT to Use This Skill - When using pre-built libraries without understanding underlying concepts - For toy problems that don't require hyperparameter tuning rigor - When domain expertise in specific problem requires different approach - If your problem doesn't require the complexity this skill provides ## Purpose and Key Concepts Hyperparameter Tuning is a critical component of the machine learning workflow. This skill covers: 1. **Theoretical foundations** — Mathematical principles and statistical concepts 2. **Practical implementation** — Working code examples and patterns 3. **Common pitfalls** — Mistakes to avoid and how to recover from them 4. **Best practices** — Industry-standard approaches and optimization techniques ## Core Workflow 1. **Understand the problem** — Clearly define what you're solving for 2. **Select approach** — Choose the right technique for your data and constraints 3. **Implement solution** — Write clean, tested code following best practices 4. **Validate results** — Verify your implementation with tests and validation 5. **Optimize performance** — Improve efficiency and accuracy incrementally ## Implementation Patterns ### Pattern 1: Basic Hyperparameter Tuning ```python import pandas as pd import numpy as np from sklearn.datasets import make_classification from sklearn.model_selection import GridSearchCV from sklearn.ensemble import RandomForestClassifier from sklearn.metrics import classification_report # Generate reproducible sample dataset X, y = make_classification(n_samples=500, n_features=10, random_state=42) # Define base estimator and parameter search space model = RandomForestClassifier(random_state=42) param_grid: dict[str, list[int]] = { 'n_estimators': [50, 100, 200] 'max_depth': [None, 10, 20] 'min_samples_split': [2, 5, 10] } # Initialize GridSearchCV with stratified cross-validation grid_search: GridSearchCV = GridSearchCV( estimator=model param_grid=param_grid cv=5 scoring='accuracy' n_jobs=-1 verbose=1 ) # Fit the model to the data grid_search.fit(X, y) # Extract and display optimal configuration best_params: dict[str, int | None] = grid_search.best_params_ best_score: float = grid_search.best_score_ print(f"Best Parameters: {best_params}") print(f"Best CV Score: {best_score:.4f}") ``` ### Pattern 2: Production-Ready Hyperparameter Tuning ```python import logging import pandas as pd import numpy as np from typing import Any, Dict, List, Tuple from sklearn.model_selection import RandomizedSearchCV from sklearn.base import BaseEstimator from sklearn.metrics import make_scorer, accuracy_score logger = logging.getLogger(__name__) class HyperparameterTuning: """Production implementation of Hyperparameter Tuning""" def __init__(self, model: BaseEstimator, param_distributions: Dict[str, List], cv: int = 5): self.model = model self.param_distributions = param_distributions self.cv = cv self.results: Dict[str, Any] = {} def execute(self, X: np.ndarray, y: np.ndarray) -> Dict[str, Any]: """Execute Hyperparameter Tuning on data""" try: logger.info("Starting hyperparameter tuning...") search = RandomizedSearchCV( estimator=self.model param_distributions=self.param_distributions n_iter=20 cv=self.cv scoring=make_scorer(accuracy_score) random_state=42 n_jobs=-1 ) search.fit(X, y) self.results = { 'best_params': search.best_params_ 'best_score': float(search.best_score_) 'cv_results_mean': search.cv_results_['mean_test_score'].tolist() 'status': 'success' } logger.info(f"Tuning completed. Best score: {self.results['best_score']:.4f}") return self.results except Exception as e: logger.error(f"Tuning failed: {str(e)}") return {'status': 'failed', 'error': str(e)} ``` ## Best Practices - ✅ Always validate your implementation on test data - ✅ Document your assumptions and methodology - ✅ Use version control for reproducibility - ✅ Monitor performance metrics in production - ✅ Periodically review and update your approach - ✅ Test with edge cases and outliers - ✅ Log all significant operations for debugging - ✅ Follow SOLID principles to keep tuning logic modular and testable - ✅ Adhere to DRY principles by abstracting repeated search configurations ## Common Pitfalls | Pitfall | Problem | Solution | | --- --- ## Constraints ### MUST DO - Use cross-validation with stratified splits for classification, group-aware splits when data has natural groups - Implement early stopping for iterative methods (XGBoost, neural nets) based on validation loss, not training loss - Search log-scale for parameters like learning_rate, C, and gamma using log-uniform distributions - Report the best configuration along with its cross-validated standard deviation to quantify result stability ### MUST NOT DO - Do not optimize hyperparameters on a single train/validation split — always use k-fold or repeated CV - Avoid exhaustive grid search when random search or Bayesian optimization would be more efficient - Never set patience too low for early stopping — 50-100 epochs minimum to allow models to learn complex patterns - Do not ignore the interaction between learning rate and batch size — they are coupled parameters ## Live References > Authoritative documentation links for this skill's domain. The model follows markdown links at load time to resolve external references and inline content. - [Scikit-learn Grid Search](https://scikit-learn.org/stable/modules/grid_search.html) - [Optuna Documentation](https://optuna.org/) - [Hyperopt — Distributed Algorithm Optimization](https://github.com/hyperopt/hyperopt/wiki/FMin) - [Bayesian Hyperparameter Optimization (Artur's Blog)](https://nikhilsood.github.io/bayesian_optimization/) - [Ray Tune Documentation](https://docs.ray.io/en/latest/tune/index.html)