How do I install the Fine-Tuning skill?

Run npx skillmds@latest add dvcrn/fine-tuning in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Fine-Tuning safe to use?

This skill has not completed SkillMD's automated safety review yet. Independent scanners report: SkillSpector: PASS, Skill Scanner: PASS. Capability flags: docs only. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Fine-Tuning?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Fine-Tuning free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Fine-Tuning?

dvcrn (@dvcrn) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Fine-Tuning by dvcrn — Agent Skill

Name: Fine-Tuning
Rating: 4.5 (4 reviews)
Author: dvcrn

File contents

When to Use

User wants to fine-tune a language model, evaluate if fine-tuning is worth it, or debug training issues.

Quick Reference

Topic	File
Provider comparison & pricing	`providers.md`
Data preparation & validation	`data-prep.md`
Training configuration	`training.md`
Evaluation & debugging	`evaluation.md`
Cost estimation & ROI	`costs.md`
Compliance & security	`compliance.md`

Core Capabilities

Decide fit — Analyze if fine-tuning beats prompting for the use case
Prepare data — Convert raw data to JSONL, deduplicate, validate format
Select provider — Compare OpenAI, Anthropic (Bedrock), Google, open source based on constraints
Estimate costs — Calculate training cost, inference savings, break-even point
Configure training — Set hyperparameters (learning rate, epochs, LoRA rank)
Run evaluation — Compare fine-tuned vs base model on task-specific metrics
Debug failures — Diagnose loss curves, overfitting, catastrophic forgetting
Handle compliance — Scan for PII, configure on-premise training, generate audit logs

Decision Checklist

Before recommending fine-tuning, ask:

What's the failure mode with prompting? (format, style, knowledge, cost)
How many training examples available? (minimum 50-100)
Expected inference volume? (affects ROI calculation)
Privacy constraints? (determines provider options)
Budget for training + ongoing inference?

Fine-Tune vs Prompt Decision

Signal	Recommendation
Format/style inconsistency	Fine-tune ✓
Missing domain knowledge	RAG first, then fine-tune if needed
High inference volume (>100K/mo)	Fine-tune for cost savings
Requirements change frequently	Stick with prompting
<50 quality examples	Prompting + few-shot

Critical Rules

Data quality > quantity — 100 great examples beat 1000 noisy ones
LoRA first — Never jump to full fine-tuning; LoRA is 10-100x cheaper
Hold out eval set — Always 80/10/10 split; never peek at test data
Same precision — Train and serve at identical precision (4-bit, 16-bit)
Baseline first — Run eval on base model before training to measure actual improvement
Expect iteration — First attempt rarely optimal; plan for 2-3 cycles

Common Pitfalls

Mistake	Fix
Training on inconsistent data	Manual review of 100+ samples before training
Learning rate too high	Start with 2e-4 for SFT, 5e-6 for RLHF
Expecting new knowledge	Fine-tuning adjusts behavior, not knowledge — use RAG
No baseline comparison	Always test base model on same eval set
Ignoring forgetting	Mix 20% general data to preserve capabilities

---name: fine-tuningdescription: Fine-tune LLMs with data preparation, provider selection, cost estimation, evaluation, and compliance checks.---## When to UseUser wants to fine-tune a language model, evaluate if fine-tuning is worth it, or debug training issues.## Quick Reference| Topic | File ||-------|------|| Provider comparison & pricing | `providers.md` || Data preparation & validation | `data-prep.md` || Training configuration | `training.md` || Evaluation & debugging | `evaluation.md` || Cost estimation & ROI | `costs.md` || Compliance & security | `compliance.md` |## Core Capabilities1. **Decide fit** — Analyze if fine-tuning beats prompting for the use case2. **Prepare data** — Convert raw data to JSONL, deduplicate, validate format3. **Select provider** — Compare OpenAI, Anthropic (Bedrock), Google, open source based on constraints4. **Estimate costs** — Calculate training cost, inference savings, break-even point5. **Configure training** — Set hyperparameters (learning rate, epochs, LoRA rank)6. **Run evaluation** — Compare fine-tuned vs base model on task-specific metrics7. **Debug failures** — Diagnose loss curves, overfitting, catastrophic forgetting8. **Handle compliance** — Scan for PII, configure on-premise training, generate audit logs## Decision ChecklistBefore recommending fine-tuning, ask:- [ ] What's the failure mode with prompting? (format, style, knowledge, cost)- [ ] How many training examples available? (minimum 50-100)- [ ] Expected inference volume? (affects ROI calculation)- [ ] Privacy constraints? (determines provider options)- [ ] Budget for training + ongoing inference?## Fine-Tune vs Prompt Decision| Signal | Recommendation ||--------|----------------|| Format/style inconsistency | Fine-tune ✓ || Missing domain knowledge | RAG first, then fine-tune if needed || High inference volume (>100K/mo) | Fine-tune for cost savings || Requirements change frequently | Stick with prompting || <50 quality examples | Prompting + few-shot |## Critical Rules- **Data quality > quantity** — 100 great examples beat 1000 noisy ones- **LoRA first** — Never jump to full fine-tuning; LoRA is 10-100x cheaper- **Hold out eval set** — Always 80/10/10 split; never peek at test data- **Same precision** — Train and serve at identical precision (4-bit, 16-bit)- **Baseline first** — Run eval on base model before training to measure actual improvement- **Expect iteration** — First attempt rarely optimal; plan for 2-3 cycles## Common Pitfalls| Mistake | Fix ||---------|-----|| Training on inconsistent data | Manual review of 100+ samples before training || Learning rate too high | Start with 2e-4 for SFT, 5e-6 for RLHF || Expecting new knowledge | Fine-tuning adjusts behavior, not knowledge — use RAG || No baseline comparison | Always test base model on same eval set || Ignoring forgetting | Mix 20% general data to preserve capabilities |

Frequently asked questions

Fine-Tuning