# LLM Finetuning

> Fine-tune LLMs with LoRA/QLoRA, instruction tuning, and alignment (DPO/ORPO).

- Skill: `aselimc/llm-finetuning` (Agent Skill)
- Install (CLI): `npx skillmds@latest add aselimc/llm-finetuning`
- Raw SKILL.md: https://api.skillmd.com/api/skills/aselimc/llm-finetuning/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: aselimc (https://skillmd.com/u/aselimc)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/aselimc/llm-finetuning

---


# LLM Fine-Tuning

## LoRA/QLoRA (Parameter-Efficient)
```python
from peft import LoraConfig, get_peft_model
config = LoraConfig(r=16, lora_alpha=32, target_modules=["q_proj", "v_proj"],
                    lora_dropout=0.05, task_type="CAUSAL_LM")
model = get_peft_model(model, config)
```

## Data Formats
- **Alpaca**: `{"instruction": ..., "input": ..., "output": ...}`
- **ShareGPT**: `{"conversations": [{"from": "human", "value": ...}, ...]}`
- **OpenAI**: `{"messages": [{"role": "user", "content": ...}, ...]}`

## Alignment
- **DPO**: direct preference optimization, no reward model needed
- **ORPO**: odds ratio preference, single-stage
- **SimPO**: simple preference optimization with reference-free margin

## Memory Optimization
- Gradient checkpointing: `model.gradient_checkpointing_enable()`
- DeepSpeed ZeRO Stage 2/3 for multi-GPU
- QLoRA: 4-bit base model + LoRA adapters

## Evaluation
Perplexity, downstream benchmarks, MT-Bench/AlpacaEval, human eval

## Key Libraries
transformers, trl, peft, unsloth, axolotl