Transformers Docs

HuggingFace Transformers Documentation

Reference for the Transformers library — the model-definition framework for state-of-the-art ML models across text, vision, audio, video, and multimodal tasks.

• GitHub | Docs | Hub
• 1M+ model checkpoints on HuggingFace Hub

CRITICAL: grep references/ for detailed docs before answering. Individual model API docs (443 models) are NOT included — refer users to the model doc index for specific model APIs.

Quick Start

from transformers import pipeline

# Inference with Pipeline
pipe = pipeline("text-generation", model="meta-llama/Llama-3-8B-Instruct")
result = pipe("Hello, how are you?", max_new_tokens=100)

# Load model + tokenizer directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3-8B-Instruct")
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3-8B-Instruct", device_map="auto")

# Fine-tuning with Trainer
from transformers import TrainingArguments, Trainer
args = TrainingArguments(output_dir="./output", num_train_epochs=3, per_device_train_batch_size=8)
trainer = Trainer(model=model, args=args, train_dataset=dataset)
trainer.train()

Core Concepts

Key Topics

Inference

• references/llm_tutorial.md — Text generation with LLMs
• references/pipeline_tutorial.md — Pipeline API tutorial
• references/generation_strategies.md — Decoding methods (greedy, beam, sampling)
• references/chat_templating.md — Chat templates
• references/conversations.md — Chat/tool use patterns

Optimization

• references/attention_interface.md — Attention backends (FlashAttention, SDPA)
• references/llm_tutorial_optimization.md — Getting the most out of LLMs
• references/continuous_batching.md — Continuous batching
• references/kv_cache.md — KV cache strategies
• references/torchcompile.md — torch.compile integration

Training

• references/trainer.md — Trainer overview
• references/training.md — Fine-tuning guide
• references/deepspeed.md — DeepSpeed integration
• references/fsdp.md — FSDP distributed training
• references/accelerate.md — Accelerate integration

Quantization

• references/quantization/overview.md — Quantization overview
• references/quantization/bitsandbytes.md — bitsandbytes (4/8-bit)
• references/quantization/gptq.md — GPTQ
• references/quantization/awq.md — AWQ
• references/quantization/gguf.md — GGUF format

Models & Preprocessors

• references/custom_models.md — Customizing models
• references/fast_tokenizers.md — Tokenizer overview
• references/image_processors.md — Image processors
• references/how_to_hack_models.md — Monkey patching models

Serving & Ecosystem

• references/serve-cli/ — Serve CLI
• references/community_integrations/vllm.md — vLLM integration
• references/community_integrations/sglang.md — SGLang integration
• references/community_integrations/llama_cpp.md — llama.cpp integration

References

• references/ — 146 doc files covering guides, API classes, quantization, serving, and ecosystem integrations
• references/main_classes/ — Core API (Trainer, Pipeline, Configuration, Tokenizer, etc.)
• references/quantization/ — 27 quantization method docs
• references/community_integrations/ — Ecosystem (vLLM, SGLang, TRL, Axolotl, etc.)