LLM Fine-Tuning
Fine-tune open-source LLMs with LoRA/QLoRA for domain-specific tasks.
Quick Start
from unsloth import FastLanguageModel
from datasets import load_dataset
from trl import SFTTrainer
# Load model with LoRA
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="unsloth/Meta-Llama-3.1-8B",
max_seq_length=2048,
load_in_4bit=True, # QLoRA
)
model = FastLanguageModel.get_peft_model(
model,
r=16,
target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
lora_alpha=16,
lora_dropout=0,
)
# Train
dataset = load_dataset("json", data_files="training_data.json")
trainer = SFTTrainer(
model=model,
tokenizer=tokenizer,
train_dataset=dataset,
dataset_text_field="text",
max_seq_length=2048,
)
trainer.train()
When to Use
- Domain-specific model adaptation
- Task-specific fine-tuning (chat, code, classification)
- Not for simple prompt engineering tasks
Step-by-Step Instructions
- Choose base model (Llama 3, Mistral, Qwen)
- Prepare training dataset in chat format
- Configure LoRA/QLoRA parameters
- Train and save adapter
Dependencies
pip install unsloth transformers datasets trl accelerate
Examples
Input: Training dataset of 1000 examples → Output: Fine-tuned LoRA adapter (50MB)
Resources
Troubleshooting
- Loss plateaus early — the learning rate is too high. Cut it by 10×
and lower batch size so validation loss keeps dropping.
- Catastrophic forgetting — mix 5–10% of the original dataset into
each epoch, or freeze the first third of the network with LoRA.
- OOM during training — use gradient accumulation,
gradient_checkpointing
(training-time), and 4-bit QLoRA quantization for consumer GPUs.
- Model regurgitates training data — you overfit. Raise dropout,
shrink epochs, and add a validation split with early stopping.
Validation
- Training loss decreases consistently
- Model generates coherent responses post-training
- Adapter merges successfully (if needed)
1---2name: llm-finetuning3description: Fine-tunes open-source LLMs (Llama, Mistral, Qwen) using LoRA/QLoRA with HuggingFace and Unsloth. Use for domain-specific model adaptation.4---5# LLM Fine-Tuning67> Fine-tune open-source LLMs with LoRA/QLoRA for domain-specific tasks.89## Quick Start10```python11from unsloth import FastLanguageModel12from datasets import load_dataset13from trl import SFTTrainer1415# Load model with LoRA16model, tokenizer = FastLanguageModel.from_pretrained(17 model_name="unsloth/Meta-Llama-3.1-8B",18 max_seq_length=2048,19 load_in_4bit=True, # QLoRA20)2122model = FastLanguageModel.get_peft_model(23 model,24 r=16,25 target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],26 lora_alpha=16,27 lora_dropout=0,28)2930# Train31dataset = load_dataset("json", data_files="training_data.json")32trainer = SFTTrainer(33 model=model,34 tokenizer=tokenizer,35 train_dataset=dataset,36 dataset_text_field="text",37 max_seq_length=2048,38)39trainer.train()40```4142## When to Use43- Domain-specific model adaptation44- Task-specific fine-tuning (chat, code, classification)45- Not for simple prompt engineering tasks4647## Step-by-Step Instructions481. Choose base model (Llama 3, Mistral, Qwen)492. Prepare training dataset in chat format503. Configure LoRA/QLoRA parameters514. Train and save adapter5253## Dependencies54```bash55pip install unsloth transformers datasets trl accelerate56```5758## Examples59Input: Training dataset of 1000 examples → Output: Fine-tuned LoRA adapter (50MB)6061## Resources62- [Unsloth](https://github.com/unslothai/unsloth)63- [HuggingFace SFT](https://huggingface.co/docs/trl/sft_trainer)64- [Examples](./examples/)6566## Troubleshooting67- **Loss plateaus early** — the learning rate is too high. Cut it by 10×68 and lower batch size so validation loss keeps dropping.69- **Catastrophic forgetting** — mix 5–10% of the original dataset into70 each epoch, or freeze the first third of the network with LoRA.71- **OOM during training** — use gradient accumulation, `gradient_checkpointing`72 (training-time), and 4-bit QLoRA quantization for consumer GPUs.73- **Model regurgitates training data** — you overfit. Raise dropout,74 shrink epochs, and add a validation split with early stopping.7576## Validation771. Training loss decreases consistently782. Model generates coherent responses post-training793. Adapter merges successfully (if needed)
ssrjkk