mkurman hydra-zen
Overview
hydra-zen is a Python-first layer on top of Hydra that removes most Hydra-specific boilerplate and eliminates hand-written YAML configs. It dynamically generates structured dataclass configs from functions, classes, and call signatures, then integrates them into Hydra workflows. Use this skill when you want typed, composable, reproducible experiment configuration without maintaining large YAML trees.
When to Use This Skill
This skill should be used when:
- You want Hydra benefits without hand-writing YAML configs
- You need type-safe, dataclass-driven experiment configuration
- You want to generate configs directly from Python call signatures
- You want to reduce Hydra boilerplate in ML/research codebases
- You need Pythonic launch/store APIs for configurable workflows
- You are building reproducible training scripts with complex parameterization
Core Capabilities
1. Installation
pip install hydra-zen
2. builds() — Generate Configs from Callables
from hydra_zen import builds
from torch.optim import AdamW
AdamWConf = builds(AdamW, lr=1e-3, weight_decay=1e-2)
# AdamWConf is a dataclass config that Hydra can instantiate
You can create configs for:
- classes
- functions
- callables
- nested object graphs
3. instantiate() — Materialize from Config
from hydra_zen import builds, instantiate
from torch.optim import AdamW
AdamWConf = builds(AdamW, lr=1e-3, weight_decay=1e-2)
optimizer = instantiate(AdamWConf)
4. make_config() — Typed Ad Hoc Configs
from hydra_zen import make_config
TrainConfig = make_config(
learning_rate=1e-3,
batch_size=64,
max_epochs=20,
model_name="resnet50",
)
cfg = TrainConfig()
print(cfg.learning_rate)
Useful when you just need a typed config object without defining a full dataclass manually.
5. store() — Register Configs with Hydra
from hydra_zen import store, builds
from torch.optim import AdamW, SGD
store(group="optimizer")(
builds(AdamW, lr=1e-3),
name="adamw"
)
store(group="optimizer")(
builds(SGD, lr=0.1, momentum=0.9),
name="sgd"
)
This gives you Hydra config-group behavior without maintaining YAML files.
6. zen() — Wrap Task Functions
from hydra_zen import zen
def train(model, optimizer, epochs: int = 10):
print(model, optimizer, epochs)
train_task = zen(train)
train_task(model="resnet50", optimizer="adamw", epochs=20)
zen() helps bridge normal Python functions and Hydra-configurable execution.
7. End-to-End Example
from hydra_zen import builds, store, zen
from torch.optim import AdamW
from torchvision.models import resnet50
ModelConf = builds(resnet50, pretrained=False, num_classes=10)
OptimConf = builds(AdamW, lr=1e-3)
store(group="model", name="resnet50")(ModelConf)
store(group="optimizer", name="adamw")(OptimConf)
@zen
def train(model, optimizer, epochs=10):
print("Model:", model)
print("Optimizer:", optimizer)
print("Epochs:", epochs)
if __name__ == "__main__":
train.hydra_main(
config_name=None,
version_base="1.3",
)
8. launch() — Programmatic Hydra Runs
from hydra_zen import builds, launch
def train(lr: float, batch_size: int):
return {"lr": lr, "batch_size": batch_size}
Conf = builds(train, lr=1e-3, batch_size=64)
job = launch(Conf)
print(job.return_value)
Useful for notebook workflows, testing, and programmatic sweep orchestration.
9. Nested Config Composition
from hydra_zen import builds, instantiate
from torch.optim import AdamW
from torchvision.models import resnet18
ModelConf = builds(resnet18, num_classes=100)
OptimConf = builds(AdamW, lr=1e-4)
ExperimentConf = builds(
dict,
model=ModelConf,
optimizer=OptimConf,
seed=42,
hydra_convert="all",
)
exp = instantiate(ExperimentConf)
print(exp["seed"])
10. ML Workflow Pattern
from hydra_zen import builds, store, zen
from pytorch_lightning import Trainer
TrainerConf = builds(
Trainer,
max_epochs=50,
accelerator="auto",
devices=1,
)
store(group="trainer", name="default")(TrainerConf)
@zen
def run_training(trainer, model, datamodule):
trainer.fit(model, datamodule)
This works especially well for:
- PyTorch Lightning
- optimizer / scheduler registries
- model family registries
- experiment launchers
- notebook-driven experimentation
Key Patterns
- Prefer
builds()over handwritten YAML for Python-heavy projects. - Use
store()to recreate Hydra config groups with less maintenance. - Use
zen()to wrap normal task functions into config-driven workflows. - Use
launch()in tests/notebooks when CLI Hydra feels heavy. - hydra-zen shines when your source of truth is Python code, not config files.