TRL Training on Hugging Face Jobs
Detailed Guide
Read the detailed guide before executing this skill. It retains the complete procedure and reference material. Treat its safety, prerequisites, and validation requirements as mandatory. For focused work, load the relevant sections; for end-to-end work, read the guide completely.
When to Use This Skill
Use this skill when users want to:
- Fine-tune language models on cloud GPUs without local infrastructure
- Train with TRL methods (SFT, DPO, GRPO, etc.)
- Run training jobs on Hugging Face Jobs infrastructure
- Convert trained models to GGUF for local deployment (Ollama, LM Studio, llama.cpp)
- Ensure trained models are permanently saved to the Hub
- Use modern workflows with optimized defaults
When to Use Unsloth
Use Unsloth (references/unsloth.md) instead of standard TRL when:
- Limited GPU memory - Unsloth uses ~60% less VRAM
- Speed matters - Unsloth is ~2x faster
- Training large models (>13B) - memory efficiency is critical
- Training Vision-Language Models (VLMs) - Unsloth has
FastVisionModel support
See references/unsloth.md for complete Unsloth documentation and scripts/unsloth_sft_example.py for a production-ready training script.
Prerequisites Checklist
Before starting any training job, verify:
✅ Account & Authentication
- Hugging Face Account with Pro, Team, or Enterprise plan (Jobs require paid plan)
- Authenticated login: Check with
hf_whoami()
- HF_TOKEN for Hub Push ⚠️ CRITICAL - Training environment is ephemeral, must push to Hub or ALL training results are lost
- Token must have write permissions
- MUST pass
secrets={"HF_TOKEN": "$HF_TOKEN"} in job config to make token available (the $HF_TOKEN syntax
references your actual token value)
✅ Dataset Requirements
- Dataset must exist on Hub or be loadable via
datasets.load_dataset()
- Format must match training method (SFT: "messages"/text/prompt-completion; DPO: chosen/rejected; GRPO: prompt-only)
- ALWAYS validate unknown datasets before GPU training to prevent format failures (see Dataset Validation section below)
- Size appropriate for hardware (Demo: 50-100 examples on t4-small; Production: 1K-10K+ on a10g-large/a100-large)
⚠️ Critical Settings
- Timeout must exceed expected training time - Default 30min is TOO SHORT for most training. Minimum recommended: 1-2 hours. Job fails and loses all progress if timeout is exceeded.
- Hub push must be enabled - Config:
push_to_hub=True, hub_model_id="username/model-name"; Job: secrets={"HF_TOKEN": "$HF_TOKEN"}
Example Training Scripts
Production-ready templates with all best practices:
Load these scripts for correctly:
scripts/train_sft_example.py - Complete SFT training with Trackio, LoRA, checkpoints
scripts/train_dpo_example.py - DPO training for preference learning
scripts/train_grpo_example.py - GRPO training for online RL
These scripts demonstrate proper Hub saving, Trackio integration, checkpoint management, and optimized parameters. Pass their content inline to hf_jobs() or use as templates for custom scripts.
Limitations
- Use this skill only when the task clearly matches its upstream product or API scope.
- Verify commands, API behavior, pricing, quotas, credentials, and deployment effects against current official documentation before making changes.
- Do not treat generated examples as a substitute for environment-specific tests, security review, or user approval for destructive or costly actions.
1---2name: hugging-face-model-trainer3description: "Train or fine-tune language and vision models using TRL (Transformer Reinforcement Learning) or Unsloth with Hugging Face Jobs infrastructure. Covers SFT, DPO, GRPO and reward modeling training methods, plus GGUF conversion for local deployment."4risk: critical5source: https://github.com/huggingface/skills/tree/main/skills/huggingface-llm-trainer6source_repo: huggingface/skills7source_type: official8date_added: 2026-07-019license: Apache-2.010license_source: https://github.com/huggingface/skills/blob/main/LICENSE11metadata:12 source: https://skillmd.com/skills/huggingface/hugging-face-model-trainer13---1415# TRL Training on Hugging Face Jobs1617## Detailed Guide1819Read [the detailed guide](references/detailed-guide.md) before executing this skill. It retains the complete procedure and reference material. Treat its safety, prerequisites, and validation requirements as mandatory. For focused work, load the relevant sections; for end-to-end work, read the guide completely.2021## When to Use This Skill2223Use this skill when users want to:24- Fine-tune language models on cloud GPUs without local infrastructure25- Train with TRL methods (SFT, DPO, GRPO, etc.)26- Run training jobs on Hugging Face Jobs infrastructure27- Convert trained models to GGUF for local deployment (Ollama, LM Studio, llama.cpp)28- Ensure trained models are permanently saved to the Hub29- Use modern workflows with optimized defaults3031### When to Use Unsloth3233Use **Unsloth** (`references/unsloth.md`) instead of standard TRL when:34- **Limited GPU memory** - Unsloth uses ~60% less VRAM35- **Speed matters** - Unsloth is ~2x faster36- Training **large models (>13B)** - memory efficiency is critical37- Training **Vision-Language Models (VLMs)** - Unsloth has `FastVisionModel` support3839See `references/unsloth.md` for complete Unsloth documentation and `scripts/unsloth_sft_example.py` for a production-ready training script.4041## Prerequisites Checklist4243Before starting any training job, verify:4445### ✅ **Account & Authentication**46- Hugging Face Account with [Pro](https://hf.co/pro), [Team](https://hf.co/enterprise), or [Enterprise](https://hf.co/enterprise) plan (Jobs require paid plan)47- Authenticated login: Check with `hf_whoami()`48- **HF_TOKEN for Hub Push** ⚠️ CRITICAL - Training environment is ephemeral, must push to Hub or ALL training results are lost49- Token must have write permissions50- **MUST pass `secrets={"HF_TOKEN": "$HF_TOKEN"}` in job config** to make token available (the `$HF_TOKEN` syntax51 references your actual token value)5253### ✅ **Dataset Requirements**54- Dataset must exist on Hub or be loadable via `datasets.load_dataset()`55- Format must match training method (SFT: "messages"/text/prompt-completion; DPO: chosen/rejected; GRPO: prompt-only)56- **ALWAYS validate unknown datasets** before GPU training to prevent format failures (see Dataset Validation section below)57- Size appropriate for hardware (Demo: 50-100 examples on t4-small; Production: 1K-10K+ on a10g-large/a100-large)5859### ⚠️ **Critical Settings**60- **Timeout must exceed expected training time** - Default 30min is TOO SHORT for most training. Minimum recommended: 1-2 hours. Job fails and loses all progress if timeout is exceeded.61- **Hub push must be enabled** - Config: `push_to_hub=True`, `hub_model_id="username/model-name"`; Job: `secrets={"HF_TOKEN": "$HF_TOKEN"}`6263## Example Training Scripts6465**Production-ready templates with all best practices:**6667Load these scripts for correctly:6869- **`scripts/train_sft_example.py`** - Complete SFT training with Trackio, LoRA, checkpoints70- **`scripts/train_dpo_example.py`** - DPO training for preference learning71- **`scripts/train_grpo_example.py`** - GRPO training for online RL7273These scripts demonstrate proper Hub saving, Trackio integration, checkpoint management, and optimized parameters. Pass their content inline to `hf_jobs()` or use as templates for custom scripts.7475## Limitations7677- Use this skill only when the task clearly matches its upstream product or API scope.78- Verify commands, API behavior, pricing, quotas, credentials, and deployment effects against current official documentation before making changes.79- Do not treat generated examples as a substitute for environment-specific tests, security review, or user approval for destructive or costly actions.