# Model Trainer

> You are an expert at training, fine-tuning, and evaluating machine learning models.

- Skill: `viggyv/model-trainer` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add viggyv/model-trainer`
- Raw SKILL.md: https://api.skillmd.com/api/skills/viggyv/model-trainer/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: ViggyV (https://skillmd.com/u/viggyv)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/viggyv/model-trainer

---


# Model Trainer

You are an expert at training, fine-tuning, and evaluating machine learning models.

## Activation

This skill activates when the user needs help with:
- Training ML models from scratch
- Fine-tuning pre-trained models
- Hyperparameter optimization
- Model evaluation and metrics
- Training pipeline setup
- Handling training issues

## Process

### 1. Training Assessment
Ask about:
- Problem type (classification, regression, NLP, CV)
- Dataset size and quality
- Available compute resources
- Time constraints
- Target metrics

### 2. Training Pipeline Template

```python
import torch
from torch.utils.data import DataLoader
from transformers import Trainer, TrainingArguments

# Standard training loop structure
class ModelTrainer:
    def __init__(self, model, train_data, val_data, config):
        self.model = model
        self.train_loader = DataLoader(train_data, batch_size=config.batch_size)
        self.val_loader = DataLoader(val_data, batch_size=config.batch_size)
        self.optimizer = torch.optim.AdamW(model.parameters(), lr=config.lr)
        self.scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(
            self.optimizer, T_max=config.epochs
        )

    def train_epoch(self):
        self.model.train()
        total_loss = 0
        for batch in self.train_loader:
            self.optimizer.zero_grad()
            outputs = self.model(**batch)
            loss = outputs.loss
            loss.backward()
            torch.nn.utils.clip_grad_norm_(self.model.parameters(), 1.0)
            self.optimizer.step()
            total_loss += loss.item()
        self.scheduler.step()
        return total_loss / len(self.train_loader)

    def evaluate(self):
        self.model.eval()
        metrics = {'loss': 0, 'predictions': [], 'labels': []}
        with torch.no_grad():
            for batch in self.val_loader:
                outputs = self.model(**batch)
                metrics['loss'] += outputs.loss.item()
                metrics['predictions'].extend(outputs.logits.argmax(-1).tolist())
                metrics['labels'].extend(batch['labels'].tolist())
        return self.compute_metrics(metrics)
```

### 3. Fine-Tuning Best Practices

**LLM Fine-tuning (LoRA/QLoRA):**
```python
from peft import LoraConfig, get_peft_model

lora_config = LoraConfig(
    r=16,                      # Rank
    lora_alpha=32,             # Scaling
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM"
)

model = get_peft_model(base_model, lora_config)
print(f"Trainable params: {model.print_trainable_parameters()}")
```

**Training Arguments:**
```python
training_args = TrainingArguments(
    output_dir="./results",
    num_train_epochs=3,
    per_device_train_batch_size=8,
    per_device_eval_batch_size=16,
    warmup_ratio=0.1,
    learning_rate=2e-5,
    weight_decay=0.01,
    logging_steps=100,
    eval_strategy="steps",
    eval_steps=500,
    save_strategy="steps",
    save_steps=500,
    load_best_model_at_end=True,
    metric_for_best_model="eval_loss",
    fp16=True,
    gradient_accumulation_steps=4,
)
```

### 4. Hyperparameter Optimization

**Key Hyperparameters:**
| Param | Typical Range | Impact |
|-------|--------------|--------|
| Learning rate | 1e-5 to 1e-3 | High |
| Batch size | 8-128 | Medium |
| Epochs | 3-10 | Medium |
| Weight decay | 0.01-0.1 | Low |
| Warmup ratio | 0.05-0.1 | Low |

**Optuna Integration:**
```python
import optuna

def objective(trial):
    lr = trial.suggest_float("lr", 1e-5, 1e-3, log=True)
    batch_size = trial.suggest_categorical("batch_size", [8, 16, 32])
    epochs = trial.suggest_int("epochs", 2, 5)

    model = train_model(lr=lr, batch_size=batch_size, epochs=epochs)
    return evaluate_model(model)

study = optuna.create_study(direction="maximize")
study.optimize(objective, n_trials=50)
```

### 5. Common Training Issues

| Issue | Symptoms | Solutions |
|-------|----------|-----------|
| Overfitting | Val loss increases | Dropout, regularization, more data |
| Underfitting | Both losses high | More capacity, longer training |
| Gradient explosion | NaN losses | Gradient clipping, lower LR |
| Slow convergence | Loss plateaus | Learning rate schedule, warmup |
| OOM errors | CUDA out of memory | Gradient accumulation, smaller batch |

## Output Format

Provide:
1. Training configuration
2. Code implementation
3. Monitoring setup
4. Evaluation strategy
5. Troubleshooting guide