Fine-Tuning on Azure AI Foundry
Fine-tune models using SFT (supervised), DPO (preference), or RFT (reinforcement with graders). Covers dataset prep, training, deployment, and evaluation.
When to Use
Use this sub-skill when the user asks about:
- Fine-tuning a model (SFT, DPO, or RFT)
- Preparing, validating, or formatting training data
- Submitting, monitoring, or diagnosing training jobs
- Calibrating graders or pass thresholds for RFT
- Deploying or evaluating a fine-tuned model
- Choosing between training types (SFT vs DPO vs RFT)
- Distillation, synthetic data generation, or dataset quality scoring
- Large file uploads for training data
- Cleaning up fine-tuning resources (files, deployments)
Do NOT use for: General model deployment without fine-tuning (use deploy-model), agent creation (use agents), prompt optimization without training (use prompt-optimizer).
Workflows
| Stage |
Guide |
| Quick start |
workflows/quickstart.md |
| Full pipeline |
workflows/full-pipeline.md |
| Create data |
workflows/dataset-creation.md |
| Iterate |
workflows/iterative-training.md |
| Diagnose |
workflows/diagnose-poor-results.md |
References
| Topic |
File |
| SFT vs DPO vs RFT |
references/training-types.md |
| Hyperparameters |
references/hyperparameters.md |
| Data formats |
references/dataset-formats.md |
| Grader design (RFT) |
references/grader-design.md |
| Reward hacking |
references/reward-hacking.md |
| Agentic RFT (tools) |
references/agentic-rft.md |
| Deployment |
references/deployment.md |
| Training curves |
references/training-curves.md |
| Evaluation |
references/evaluation.md |
| Vision fine-tuning |
references/vision-fine-tuning.md |
| Large file uploads |
references/large-file-uploads.md |
| Platform gotchas |
references/platform-gotchas.md |
Scripts
| Script |
Purpose |
scripts/submit_training.py |
Submit SFT/DPO/RFT jobs |
scripts/monitor_training.py |
Poll job until completion |
scripts/calibrate_grader.py |
Find optimal RFT pass_threshold |
scripts/check_training.py |
Analyze curves, list checkpoints |
scripts/deploy_model.py |
Deploy via ARM REST API |
scripts/evaluate_model.py |
LLM judge evaluation |
scripts/convert_dataset.py |
Convert between SFT/DPO/RFT formats |
scripts/generate_distillation_data.py |
Generate synthetic training data |
scripts/score_dataset.py |
Quality scoring on training data |
scripts/cleanup.py |
Delete old files and deployments |
scripts/validate/ |
Data validators (SFT, DPO, RFT) + stats |
Rules
- Always baseline first — evaluate the base model before fine-tuning
- Validate data before submitting — run
scripts/validate/validate_sft.py
- Calibrate RFT graders — target 25-50% failure rate on the base model
- Evaluate checkpoints — don't blindly deploy the final one
- Measure token cost alongside accuracy when comparing models
Quick Reference
| Task |
Command |
| Validate SFT data |
python scripts/validate/validate_sft.py data.jsonl |
| Submit SFT job |
python scripts/submit_training.py --model gpt-4.1-mini --training-file train.jsonl --validation-file val.jsonl --type sft |
| Monitor job |
python scripts/monitor_training.py --job-id ftjob-xxx |
| Analyze curves |
python scripts/check_training.py --job-id ftjob-xxx |
| Deploy model |
python scripts/deploy_model.py --model-id ft:gpt-4.1-mini:... --name my-eval |
| Evaluate model |
python scripts/evaluate_model.py --deployment-name my-eval --test-file test.jsonl |
Error Handling
| Error |
Cause |
Fix |
| "API version not supported" |
Older openai SDK on /v1/ endpoint |
Upgrade to openai>=1.0 |
| "does not support fine-tuning with Standard TrainingType" |
OSS model needs globalStandard |
Use --use-rest flag or script auto-falls back |
| Job stuck in post-training eval |
Under-provisioned tool endpoint (RFT) |
Scale to S2+, enable Always On |
| "DeploymentNotReady" after ARM succeeds |
ARM/data-plane race condition |
Delete and recreate deployment, wait 5 min |
| Content safety block at deployment |
PII-dense training data |
Remove problematic document types |
1---2name: finetuning3description: "Fine-tune models on Azure AI Foundry using SFT (supervised), DPO (preference), or RFT (reinforcement with graders). Covers dataset preparation, training job submission, deployment, and evaluation. USE FOR: fine-tune, SFT, DPO, RFT, training data, grader, distillation, fine-tuned model, training job, large file upload, calibrate grader, deploy fine-tuned model, evaluate fine-tuned model. DO NOT USE FOR: general model deployment without fine-tuning (use deploy-model), agent creation (use agents), prompt optimization without training (use prompt-optimizer)."4license: MIT5metadata:6 source: https://skillmd.com/skills/microsoft/finetuning7 author: Microsoft8 version: "0.0.0-placeholder"9---1011# Fine-Tuning on Azure AI Foundry1213Fine-tune models using SFT (supervised), DPO (preference), or RFT (reinforcement with graders). Covers dataset prep, training, deployment, and evaluation.1415## When to Use1617Use this sub-skill when the user asks about:18- Fine-tuning a model (SFT, DPO, or RFT)19- Preparing, validating, or formatting training data20- Submitting, monitoring, or diagnosing training jobs21- Calibrating graders or pass thresholds for RFT22- Deploying or evaluating a fine-tuned model23- Choosing between training types (SFT vs DPO vs RFT)24- Distillation, synthetic data generation, or dataset quality scoring25- Large file uploads for training data26- Cleaning up fine-tuning resources (files, deployments)2728**Do NOT use for:** General model deployment without fine-tuning (use deploy-model), agent creation (use agents), prompt optimization without training (use prompt-optimizer).2930## Workflows3132| Stage | Guide |33|-------|-------|34| **Quick start** | [workflows/quickstart.md](workflows/quickstart.md) |35| **Full pipeline** | [workflows/full-pipeline.md](workflows/full-pipeline.md) |36| **Create data** | [workflows/dataset-creation.md](workflows/dataset-creation.md) |37| **Iterate** | [workflows/iterative-training.md](workflows/iterative-training.md) |38| **Diagnose** | [workflows/diagnose-poor-results.md](workflows/diagnose-poor-results.md) |3940## References4142| Topic | File |43|-------|------|44| SFT vs DPO vs RFT | [references/training-types.md](references/training-types.md) |45| Hyperparameters | [references/hyperparameters.md](references/hyperparameters.md) |46| Data formats | [references/dataset-formats.md](references/dataset-formats.md) |47| Grader design (RFT) | [references/grader-design.md](references/grader-design.md) |48| Reward hacking | [references/reward-hacking.md](references/reward-hacking.md) |49| Agentic RFT (tools) | [references/agentic-rft.md](references/agentic-rft.md) |50| Deployment | [references/deployment.md](references/deployment.md) |51| Training curves | [references/training-curves.md](references/training-curves.md) |52| Evaluation | [references/evaluation.md](references/evaluation.md) |53| Vision fine-tuning | [references/vision-fine-tuning.md](references/vision-fine-tuning.md) |54| Large file uploads | [references/large-file-uploads.md](references/large-file-uploads.md) |55| Platform gotchas | [references/platform-gotchas.md](references/platform-gotchas.md) |5657## Scripts5859| Script | Purpose |60|--------|---------|61| `scripts/submit_training.py` | Submit SFT/DPO/RFT jobs |62| `scripts/monitor_training.py` | Poll job until completion |63| `scripts/calibrate_grader.py` | Find optimal RFT pass_threshold |64| `scripts/check_training.py` | Analyze curves, list checkpoints |65| `scripts/deploy_model.py` | Deploy via ARM REST API |66| `scripts/evaluate_model.py` | LLM judge evaluation |67| `scripts/convert_dataset.py` | Convert between SFT/DPO/RFT formats |68| `scripts/generate_distillation_data.py` | Generate synthetic training data |69| `scripts/score_dataset.py` | Quality scoring on training data |70| `scripts/cleanup.py` | Delete old files and deployments |71| `scripts/validate/` | Data validators (SFT, DPO, RFT) + stats |7273## Rules74751. **Always baseline first** — evaluate the base model before fine-tuning762. **Validate data** before submitting — run `scripts/validate/validate_sft.py`773. **Calibrate RFT graders** — target 25-50% failure rate on the base model784. **Evaluate checkpoints** — don't blindly deploy the final one795. **Measure token cost** alongside accuracy when comparing models8081## Quick Reference8283| Task | Command |84|------|---------|85| Validate SFT data | `python scripts/validate/validate_sft.py data.jsonl` |86| Submit SFT job | `python scripts/submit_training.py --model gpt-4.1-mini --training-file train.jsonl --validation-file val.jsonl --type sft` |87| Monitor job | `python scripts/monitor_training.py --job-id ftjob-xxx` |88| Analyze curves | `python scripts/check_training.py --job-id ftjob-xxx` |89| Deploy model | `python scripts/deploy_model.py --model-id ft:gpt-4.1-mini:... --name my-eval` |90| Evaluate model | `python scripts/evaluate_model.py --deployment-name my-eval --test-file test.jsonl` |9192## Error Handling9394| Error | Cause | Fix |95|-------|-------|-----|96| "API version not supported" | Older `openai` SDK on `/v1/` endpoint | Upgrade to `openai>=1.0` |97| "does not support fine-tuning with Standard TrainingType" | OSS model needs `globalStandard` | Use `--use-rest` flag or script auto-falls back |98| Job stuck in post-training eval | Under-provisioned tool endpoint (RFT) | Scale to S2+, enable Always On |99| "DeploymentNotReady" after ARM succeeds | ARM/data-plane race condition | Delete and recreate deployment, wait 5 min |100| Content safety block at deployment | PII-dense training data | Remove problematic document types |