Experiment Tracking
Track ML experiments, metrics, and models.
Comparison
| Platform |
Best For |
Self-hosted |
Visualization |
| MLflow |
Open-source, model registry |
Yes |
Basic |
| W&B |
Collaboration, sweeps |
Limited |
Excellent |
| Neptune |
Team collaboration |
No |
Good |
| ClearML |
Full MLOps |
Yes |
Good |
MLflow
Open-source platform from Databricks.
Core components:
- Tracking: Log parameters, metrics, artifacts
- Projects: Reproducible runs (MLproject file)
- Models: Package and deploy models
- Registry: Model versioning and staging
Strengths: Self-hosted, open-source, model registry, framework integrations
Limitations: Basic visualization, less collaborative features
Key concept: Autologging for major frameworks - automatic metric capture with one line.
Weights & Biases (W&B)
Cloud-first experiment tracking with excellent visualization.
Core features:
- Experiment tracking: Metrics, hyperparameters, system stats
- Sweeps: Hyperparameter search (grid, random, Bayesian)
- Artifacts: Dataset and model versioning
- Reports: Shareable documentation
Strengths: Beautiful visualizations, team collaboration, hyperparameter sweeps
Limitations: Cloud-dependent, limited self-hosting
Key concept: wandb.init() + wandb.log() - simple API, powerful features.
What to Track
| Category |
Examples |
| Hyperparameters |
Learning rate, batch size, architecture |
| Metrics |
Loss, accuracy, F1, per-epoch values |
| Artifacts |
Model checkpoints, configs, datasets |
| System |
GPU usage, memory, runtime |
| Code |
Git commit, diff, requirements |
Model Registry Concepts
| Stage |
Purpose |
| None |
Just logged, not registered |
| Staging |
Testing, validation |
| Production |
Serving live traffic |
| Archived |
Deprecated, kept for reference |
Decision Guide
| Scenario |
Recommendation |
| Self-hosted requirement |
MLflow |
| Team collaboration |
W&B |
| Model registry focus |
MLflow |
| Hyperparameter sweeps |
W&B |
| Beautiful dashboards |
W&B |
| Full MLOps pipeline |
MLflow + deployment tools |
Resources
1---2name: experiment-tracking3description: Use when "experiment tracking", "MLflow", "Weights & Biases", "wandb", "model registry", "hyperparameter logging", "ML experiments", "training metrics"4---5
6# Experiment Tracking
7
8Track ML experiments, metrics, and models.
9
10## Comparison
11
12| Platform | Best For | Self-hosted | Visualization |
13|----------|----------|-------------|---------------|
14| **MLflow** | Open-source, model registry | Yes | Basic |
15| **W&B** | Collaboration, sweeps | Limited | Excellent |
16| **Neptune** | Team collaboration | No | Good |
17| **ClearML** | Full MLOps | Yes | Good |
18
19---
20
21## MLflow
22
23Open-source platform from Databricks.
24
25**Core components:**
26
27- **Tracking**: Log parameters, metrics, artifacts
28- **Projects**: Reproducible runs (MLproject file)
29- **Models**: Package and deploy models
30- **Registry**: Model versioning and staging
31
32**Strengths**: Self-hosted, open-source, model registry, framework integrations
33**Limitations**: Basic visualization, less collaborative features
34
35**Key concept**: Autologging for major frameworks - automatic metric capture with one line.
36
37---
38
39## Weights & Biases (W&B)
40
41Cloud-first experiment tracking with excellent visualization.
42
43**Core features:**
44
45- **Experiment tracking**: Metrics, hyperparameters, system stats
46- **Sweeps**: Hyperparameter search (grid, random, Bayesian)
47- **Artifacts**: Dataset and model versioning
48- **Reports**: Shareable documentation
49
50**Strengths**: Beautiful visualizations, team collaboration, hyperparameter sweeps
51**Limitations**: Cloud-dependent, limited self-hosting
52
53**Key concept**: `wandb.init()` + `wandb.log()` - simple API, powerful features.
54
55---
56
57## What to Track
58
59| Category | Examples |
60|----------|----------|
61| **Hyperparameters** | Learning rate, batch size, architecture |
62| **Metrics** | Loss, accuracy, F1, per-epoch values |
63| **Artifacts** | Model checkpoints, configs, datasets |
64| **System** | GPU usage, memory, runtime |
65| **Code** | Git commit, diff, requirements |
66
67---
68
69## Model Registry Concepts
70
71| Stage | Purpose |
72|-------|---------|
73| **None** | Just logged, not registered |
74| **Staging** | Testing, validation |
75| **Production** | Serving live traffic |
76| **Archived** | Deprecated, kept for reference |
77
78---
79
80## Decision Guide
81
82| Scenario | Recommendation |
83|----------|----------------|
84| Self-hosted requirement | MLflow |
85| Team collaboration | W&B |
86| Model registry focus | MLflow |
87| Hyperparameter sweeps | W&B |
88| Beautiful dashboards | W&B |
89| Full MLOps pipeline | MLflow + deployment tools |
90
91## Resources
92
93- MLflow: <https://mlflow.org/docs/latest/>
94- W&B: <https://docs.wandb.ai/>
eyadsibai