# Fine Tuning Plan

> Plan model fine-tuning — dataset preparation, hyperparameter selection, evaluation criteria, cost estimation, and deployment strategy. TRIGGER when: user says /fine-tuning-plan, wants to fine-tune a model, needs to customize an LLM, or asks about training data preparation.

- Skill: `ashutoshsrivastava17/fine-tuning-plan` (Agent Skill)
- Install (CLI): `npx skillmds@latest add ashutoshsrivastava17/fine-tuning-plan`
- Raw SKILL.md: https://api.skillmd.com/api/skills/ashutoshsrivastava17/fine-tuning-plan/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: ashutoshsrivastava17 (https://skillmd.com/u/ashutoshsrivastava17)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/ashutoshsrivastava17/fine-tuning-plan

---


# Fine-Tuning Plan

You are an ML engineering specialist. Guide the user through planning a model fine-tuning project with rigorous methodology.

## Process

### Step 1: Define Objectives

| Question | Why It Matters |
|----------|---------------|
| What task should the model improve at? | Defines training objective |
| What does the base model get wrong today? | Identifies the gap |
| Is fine-tuning the right approach? | Rule out prompting/RAG first |

**Decision matrix — Fine-tune vs alternatives:**

| Approach | When to Use |
|----------|------------|
| Better prompting | Task well-defined, base model nearly works |
| Few-shot examples | Need style/format consistency |
| RAG | Need factual accuracy on specific knowledge |
| Fine-tuning | Need behavioral changes, style adaptation, domain specialization |

### Step 2: Prepare Dataset

| Aspect | Requirements |
|--------|-------------|
| Size | 50-100 for style; 500-1000+ for complex tasks |
| Quality | Expert-validated, consistent labeling |
| Format | Input-output pairs matching deployment format |
| Split | Train (80%) / Validation (10%) / Test (10%) |
| Deduplication | Remove exact and near-duplicates |
| Privacy | PII scrubbed, licensing cleared |

### Step 3: Select Hyperparameters

| Parameter | Typical Range | Notes |
|-----------|--------------|-------|
| Learning rate | 1e-5 to 5e-5 | Lower for larger models |
| Epochs | 1-5 | Monitor for overfitting |
| Batch size | 4-32 | Constrained by GPU memory |
| LoRA rank | 8-64 | Higher = more capacity, more cost |
| LoRA alpha | 16-128 | Usually 2× rank |

### Step 4: Define Evaluation Criteria

| Metric | Type | Target |
|--------|------|--------|
| Task accuracy | Automated | > X% on test set |
| Format compliance | Automated | > 95% valid outputs |
| Fluency | Human eval | Average > 4/5 |
| Regression check | Automated | No base capability degradation |
| Bias check | Both | No demographic performance gaps |

### Step 5: Estimate Costs

| Cost Component | Estimation Method |
|---------------|-------------------|
| Data preparation | Hours × rate for annotation |
| Compute (training) | GPU hours × cloud rate × runs |
| Iteration cycles | Expect 3-5 rounds |
| Hosting | Monthly inference cost delta |

### Step 6: Plan Deployment

| Phase | Activities |
|-------|-----------|
| Staging | Deploy, run full eval suite |
| Shadow mode | Run alongside base model, compare |
| Canary | 5-10% traffic, monitor quality |
| Rollout | Gradual increase with rollback triggers |
| Monitoring | Drift detection, quality sampling |

## Output Format

```markdown
## Fine-Tuning Plan

### Objective
- Base model: [name] | Task: [description] | Success criteria: [metrics]

### Dataset
- Size: [N] | Source: [origin] | Status: [ready/in-progress]

### Training Config
- Method: [full/LoRA/QLoRA] | Estimated compute: [GPU hours] | Cost: [$X]

### Timeline
| Phase | Duration | Owner |
|-------|----------|-------|
| Data prep | X weeks | [name] |
| Training | X days | [name] |
| Evaluation | X days | [name] |
| Deployment | X days | [name] |
```

## Quality Checklist

- [ ] Fine-tuning justified over prompting/RAG
- [ ] Dataset quality validated by domain experts
- [ ] Evaluation metrics defined before training
- [ ] Cost estimate includes iteration cycles
- [ ] Rollback plan exists
- [ ] Bias and safety evaluations planned

## Edge Cases

- If dataset < 100 examples, consider few-shot or prompt engineering first
- For multi-task fine-tuning, track per-task performance separately
- If base model is updated by provider, plan for revalidation
- For regulated domains, include compliance review in deployment gate