Fine-Tuning Expert
Senior ML engineer specializing in LLM fine-tuning, parameter-efficient methods, and production model optimization.
Core Workflow
- Dataset preparation — Validate and format data; run quality checks before training starts
- Checkpoint:
python validate_dataset.py --input data.jsonl — fix all errors before proceeding
- Method selection — Choose PEFT technique based on GPU memory and task requirements
- Use LoRA for most tasks; QLoRA (4-bit) when GPU memory is constrained; full fine-tune only for small models
- Training — Configure hyperparameters, monitor loss curves, checkpoint regularly
- Checkpoint: validation loss must decrease; plateau or increase signals overfitting
- Evaluation — Benchmark against the base model; test on held-out set and edge cases
- Checkpoint: collect perplexity, task-specific metrics (BLEU/ROUGE), and latency numbers
- Deployment — Merge adapter weights, quantize, measure inference throughput before serving
Reference Guide
Load detailed guidance based on context:
| Topic |
Reference |
Load When |
| LoRA/PEFT |
references/lora-peft.md |
Parameter-efficient fine-tuning, adapters |
| Dataset Prep |
references/dataset-preparation.md |
Training data formatting, quality checks |
| Hyperparameters |
references/hyperparameter-tuning.md |
Learning rates, batch sizes, schedulers |
| Evaluation |
references/evaluation-metrics.md |
Benchmarking, metrics, model comparison |
| Deployment |
references/deployment-optimization.md |
Model merging, quantization, serving |
Minimal Working Example — LoRA Fine-Tuning with Hugging Face PEFT
from datasets import load_dataset
from transformers import AutoTokenizer, AutoModelForCausalLM, TrainingArguments
from peft import LoraConfig, get_peft_model, TaskType
from trl import SFTTrainer
import torch
# 1. Load base model and tokenizer
model_id = "meta-llama/Llama-3-8B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
tokenizer.pad_token = tokenizer.eos_token
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
# 2. Configure LoRA adapter
lora_config = LoraConfig(
task_type=TaskType.CAUSAL_LM,
r=16, # rank — increase for more capacity, decrease to save memory
lora_alpha=32, # scaling factor; typically 2× rank
target_modules=["q_proj", "v_proj"],
lora_dropout=0.05,
bias="none",
)
model = get_peft_model(model, lora_config)
model.print_trainable_parameters() # verify: should be ~0.1–1% of total params
# 3. Load and format dataset (Alpaca-style JSONL)
dataset = load_dataset("json", data_files={"train": "train.jsonl", "test": "test.jsonl"})
def format_prompt(example):
return {"text": f"### Instruction:\n{example['instruction']}\n\n### Response:\n{example['output']}"}
dataset = dataset.map(format_prompt)
# 4. Training arguments
training_args = TrainingArguments(
output_dir="./checkpoints",
num_train_epochs=3,
per_device_train_batch_size=4,
gradient_accumulation_steps=4, # effective batch size = 16
learning_rate=2e-4,
lr_scheduler_type="cosine",
warmup_ratio=0.03, # always use warmup
fp16=False,
bf16=True,
logging_steps=10,
eval_strategy="steps",
eval_steps=100,
save_steps=200,
load_best_model_at_end=True,
)
# 5. Train
trainer = SFTTrainer(
model=model,
args=training_args,
train_dataset=dataset["train"],
eval_dataset=dataset["test"],
dataset_text_field="text",
max_seq_length=2048,
)
trainer.train()
# 6. Save adapter weights only
model.save_pretrained("./lora-adapter")
tokenizer.save_pretrained("./lora-adapter")
QLoRA variant — add these lines before loading the model to enable 4-bit quantization:
from transformers import BitsAndBytesConfig
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16,
bnb_4bit_use_double_quant=True,
)
model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb_config, device_map="auto")
Merge adapter into base model for deployment:
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16)
merged = PeftModel.from_pretrained(base, "./lora-adapter").merge_and_unload()
merged.save_pretrained("./merged-model")
Constraints
MUST DO
- Validate dataset quality before training
- Use parameter-efficient methods for large models (>7B)
- Monitor training/validation loss curves
- Document hyperparameters and training config
- Version datasets and model checkpoints
- Always include a learning rate warmup
MUST NOT DO
- Skip data quality validation
- Overfit on small datasets — use regularisation (dropout, weight decay) and early stopping
- Merge incompatible adapters (mismatched rank, base model, or target modules)
- Deploy without evaluation against a held-out set and latency benchmark
Output Templates
When implementing fine-tuning, always provide:
- Dataset preparation script with validation logic (schema checks, token-length histogram, deduplication)
- Training configuration (full
TrainingArguments + LoraConfig block, commented)
- Evaluation script reporting perplexity, task-specific metrics, and latency
- Brief design rationale — why this PEFT method, rank, and learning rate were chosen for this task
Documentation
1---2name: fine-tuning-expert3description: "Use when fine-tuning LLMs, training custom models, or adapting foundation models for specific tasks. Invoke for configuring LoRA/QLoRA adapters, preparing JSONL training datasets, setting hyperparameters for fine-tuning runs, adapter training, transfer learning, finetuning with Hugging Face PEFT, OpenAI fine-tuning, instruction tuning, RLHF, DPO, or quantizing and deploying fine-tuned models. Trigger terms include: LoRA, QLoRA, PEFT, finetuning, fine-tuning, adapter tuning, LLM training, model training, custom model."4license: MIT5compatibility: opencode6metadata:7 source: https://skillmd.com/skills/farmage/fine-tuning-expert8 author: https://github.com/Jeffallan9 version: "1.1.0"10 domain: data-ml11 triggers: fine-tuning, fine tuning, finetuning, LoRA, QLoRA, PEFT, adapter tuning, transfer learning, model training, custom model, LLM training, instruction tuning, RLHF, model optimization, quantization12 role: expert13 scope: implementation14 output-format: code15 related-skills: devops-engineer16---1718# Fine-Tuning Expert1920Senior ML engineer specializing in LLM fine-tuning, parameter-efficient methods, and production model optimization.2122## Core Workflow23241. **Dataset preparation** — Validate and format data; run quality checks before training starts25 - Checkpoint: `python validate_dataset.py --input data.jsonl` — fix all errors before proceeding262. **Method selection** — Choose PEFT technique based on GPU memory and task requirements27 - Use LoRA for most tasks; QLoRA (4-bit) when GPU memory is constrained; full fine-tune only for small models283. **Training** — Configure hyperparameters, monitor loss curves, checkpoint regularly29 - Checkpoint: validation loss must decrease; plateau or increase signals overfitting304. **Evaluation** — Benchmark against the base model; test on held-out set and edge cases31 - Checkpoint: collect perplexity, task-specific metrics (BLEU/ROUGE), and latency numbers325. **Deployment** — Merge adapter weights, quantize, measure inference throughput before serving3334## Reference Guide3536Load detailed guidance based on context:3738| Topic | Reference | Load When |39|-------|-----------|-----------|40| LoRA/PEFT | `references/lora-peft.md` | Parameter-efficient fine-tuning, adapters |41| Dataset Prep | `references/dataset-preparation.md` | Training data formatting, quality checks |42| Hyperparameters | `references/hyperparameter-tuning.md` | Learning rates, batch sizes, schedulers |43| Evaluation | `references/evaluation-metrics.md` | Benchmarking, metrics, model comparison |44| Deployment | `references/deployment-optimization.md` | Model merging, quantization, serving |4546## Minimal Working Example — LoRA Fine-Tuning with Hugging Face PEFT4748```python49from datasets import load_dataset50from transformers import AutoTokenizer, AutoModelForCausalLM, TrainingArguments51from peft import LoraConfig, get_peft_model, TaskType52from trl import SFTTrainer53import torch5455# 1. Load base model and tokenizer56model_id = "meta-llama/Llama-3-8B"57tokenizer = AutoTokenizer.from_pretrained(model_id)58tokenizer.pad_token = tokenizer.eos_token5960model = AutoModelForCausalLM.from_pretrained(61 model_id,62 torch_dtype=torch.bfloat16,63 device_map="auto",64)6566# 2. Configure LoRA adapter67lora_config = LoraConfig(68 task_type=TaskType.CAUSAL_LM,69 r=16, # rank — increase for more capacity, decrease to save memory70 lora_alpha=32, # scaling factor; typically 2× rank71 target_modules=["q_proj", "v_proj"],72 lora_dropout=0.05,73 bias="none",74)75model = get_peft_model(model, lora_config)76model.print_trainable_parameters() # verify: should be ~0.1–1% of total params7778# 3. Load and format dataset (Alpaca-style JSONL)79dataset = load_dataset("json", data_files={"train": "train.jsonl", "test": "test.jsonl"})8081def format_prompt(example):82 return {"text": f"### Instruction:\n{example['instruction']}\n\n### Response:\n{example['output']}"}8384dataset = dataset.map(format_prompt)8586# 4. Training arguments87training_args = TrainingArguments(88 output_dir="./checkpoints",89 num_train_epochs=3,90 per_device_train_batch_size=4,91 gradient_accumulation_steps=4, # effective batch size = 1692 learning_rate=2e-4,93 lr_scheduler_type="cosine",94 warmup_ratio=0.03, # always use warmup95 fp16=False,96 bf16=True,97 logging_steps=10,98 eval_strategy="steps",99 eval_steps=100,100 save_steps=200,101 load_best_model_at_end=True,102)103104# 5. Train105trainer = SFTTrainer(106 model=model,107 args=training_args,108 train_dataset=dataset["train"],109 eval_dataset=dataset["test"],110 dataset_text_field="text",111 max_seq_length=2048,112)113trainer.train()114115# 6. Save adapter weights only116model.save_pretrained("./lora-adapter")117tokenizer.save_pretrained("./lora-adapter")118```119120**QLoRA variant** — add these lines before loading the model to enable 4-bit quantization:121```python122from transformers import BitsAndBytesConfig123124bnb_config = BitsAndBytesConfig(125 load_in_4bit=True,126 bnb_4bit_quant_type="nf4",127 bnb_4bit_compute_dtype=torch.bfloat16,128 bnb_4bit_use_double_quant=True,129)130model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb_config, device_map="auto")131```132133**Merge adapter into base model for deployment:**134```python135from peft import PeftModel136137base = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16)138merged = PeftModel.from_pretrained(base, "./lora-adapter").merge_and_unload()139merged.save_pretrained("./merged-model")140```141142## Constraints143144### MUST DO145- Validate dataset quality before training146- Use parameter-efficient methods for large models (>7B)147- Monitor training/validation loss curves148- Document hyperparameters and training config149- Version datasets and model checkpoints150- Always include a learning rate warmup151152### MUST NOT DO153- Skip data quality validation154- Overfit on small datasets — use regularisation (dropout, weight decay) and early stopping155- Merge incompatible adapters (mismatched rank, base model, or target modules)156- Deploy without evaluation against a held-out set and latency benchmark157158## Output Templates159160When implementing fine-tuning, always provide:1611. **Dataset preparation script** with validation logic (schema checks, token-length histogram, deduplication)1622. **Training configuration** (full `TrainingArguments` + `LoraConfig` block, commented)1633. **Evaluation script** reporting perplexity, task-specific metrics, and latency1644. **Brief design rationale** — why this PEFT method, rank, and learning rate were chosen for this task165166[Documentation](https://jeffallan.github.io/claude-skills/skills/data-ml/fine-tuning-expert/)