# Hyperparameter Search

> Systematic hyperparameter optimization with Optuna. Bayesian search with pruning.

- Skill: `aselimc/hyperparameter-search` (Agent Skill)
- Install (CLI): `npx skillmds@latest add aselimc/hyperparameter-search`
- Raw SKILL.md: https://api.skillmd.com/api/skills/aselimc/hyperparameter-search/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: aselimc (https://skillmd.com/u/aselimc)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/aselimc/hyperparameter-search

---


# Hyperparameter Search

## Optuna Pattern
```python
import optuna

def objective(trial):
    lr = trial.suggest_float("lr", 1e-5, 1e-2, log=True)
    bs = trial.suggest_categorical("batch_size", [16, 32, 64])
    wd = trial.suggest_float("weight_decay", 1e-6, 1e-2, log=True)

    model = build_model(trial)
    val_metric = train_and_eval(model, lr, bs, wd)

    trial.report(val_metric, step=epoch)  # for pruning
    if trial.should_prune(): raise optuna.TrialPruned()
    return val_metric

study = optuna.create_study(direction="maximize",
    pruner=optuna.pruners.MedianPruner(n_warmup_steps=5))
study.optimize(objective, n_trials=100)
print(study.best_params)
```

## Rules
- Use log-uniform for learning rates
- Use categorical for architecture choices
- Enable pruning (MedianPruner) to save compute
- Report best config reproducibly (save full config, not just best params)
- For multi-objective (accuracy vs latency): `create_study(directions=["maximize", "minimize"])`

## Key Libraries
optuna, ray[tune]