# Optimization Qlora

> QLoRA fine-tuning for DSPy programs

- Skill: `j33bs/optimization-qlora` (Agent Skill)
- Install (CLI): `npx skillmds@latest add j33bs/optimization-qlora`
- Raw SKILL.md: https://api.skillmd.com/api/skills/j33bs/optimization-qlora/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: j33bs (https://skillmd.com/u/j33bs)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/j33bs/optimization-qlora

---


# QLoRA Fine-Tuning

## 🎯 Trigger Conditions
Use when asked about QLoRA (Quantized Low-Rank Adaptation), memory-efficient fine-tuning, or parameter-efficient tuning for DSPy.

## 📚 Prerequisites
- `peft` package installed
- `bitsandbytes` package installed
- GPU with sufficient VRAM
- Training data prepared

## 🛠️ QLoRA Implementation

### 1. Basic QLoRA Setup
```python
from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training
from transformers import BitsAndBytesConfig

# Configure quantization
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype="float16",
    bnb_4bit_use_double_quant=True
)

# Load model with quantization
model = AutoModelForCausalLM.from_pretrained(
    "your-model",
    quantization_config=bnb_config,
    device_map="auto"
)

# Prepare for training
model = prepare_model_for_kbit_training(model)

# Apply LoRA
lora_config = LoraConfig(
    r=16,
    lora_alpha=32,
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05,
    bias="none"
)

model = get_peft_model(model, lora_config)
```

### 2. QLoRA for DSPy
```python
import dspy
from peft import LoraConfig, get_peft_model

# Load DSPy program
program = dspy.ChainOfThought("question -> reasoning -> answer")

# Apply QLoRA to underlying model
lora_config = LoraConfig(
    r=8,
    lora_alpha=16,
    target_modules=["query", "value"],
    lora_dropout=0.1
)

program.model = get_peft_model(program.model, lora_config)
```

### 3. QLoRA Training
```python
from transformers import TrainingArguments, Trainer

# Configure training
training_args = TrainingArguments(
    output_dir="qlora-output",
    num_train_epochs=3,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,
    learning_rate=2e-4,
    fp16=True,
    logging_steps=10
)

# Create trainer
trainer = Trainer(
    model=program.model,
    args=training_args,
    train_dataset=train_dataset,
    tokenizer=tokenizer
)

# Train
trainer.train()
```

### 4. QLoRA Configuration Options
```python
lora_config = LoraConfig(
    # Rank
    r=8,  # Low-rank dimension
    
    # Alpha
    lora_alpha=16,  # Scaling factor
    
    # Target modules
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
    
    # Dropout
    lora_dropout=0.05,
    
    # Bias
    bias="none",
    
    # Task type
    task_type="CAUSAL_LM"
)
```

## ⚠️ Pitfalls
- **VRAM requirements**: Still requires significant VRAM
- **Quality trade-off**: Quantization affects quality
- **Training stability**: QLoRA can be unstable
- **Compatibility**: Not all models support QLoRA

## 📖 References
- [QLoRA Paper](https://arxiv.org/abs/2305.14314)
- [PEFT QLoRA](https://huggingface.co/docs/peft/main/en/conceptual_guides/qlora)