# Transformers Knowledge Patch > Transformers - Skill: `nevaberry/transformers-knowledge-patch` (Agent Skill, multi-file: 8 files) - Install (CLI): `npx skillmds@latest add nevaberry/transformers-knowledge-patch` - Raw SKILL.md: https://api.skillmd.com/api/skills/nevaberry/transformers-knowledge-patch/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - License: MIT - Author: nevaberry (https://skillmd.com/u/nevaberry) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/nevaberry/transformers-knowledge-patch --- # Transformers Knowledge Patch Load this skill before changing Transformers applications, integrations, custom models, tokenizers, processors, training loops, or serving code. Determine the installed Transformers version first and apply only guidance introduced at or below that version. Prefer the project's manifests, code, and tests whenever they disagree with this patch. ## Reference index | Reference | Topics | | --- | --- | | [Compatibility and API migration](references/compatibility-and-api-migration.md) | Runtime floors, removals, tokenizers, configuration, renamed and changed APIs | | [Generation, caches, and serving](references/generation-caches-and-serving.md) | Generation contracts, cache behavior, continuous batching, chat CLI, local serving | | [Loading, quantization, and kernels](references/loading-quantization-and-kernels.md) | Checkpoint loading, quantizers, attention backends, custom kernels, serialization | | [Model and task integrations](references/model-and-task-integrations.md) | Language, multimodal, vision, audio, document, scientific, and robotics architectures | | [Multimodal processing and pipelines](references/multimodal-processing-and-pipelines.md) | Processors, chat templates, media inputs, pipeline behavior, visualization | | [Training and distributed execution](references/training-and-distributed-execution.md) | Trainer behavior, tensor/expert/sequence parallelism, FSDP, optimizers, backends | ## Start with the breaking changes ### Update runtime dependencies - Transformers 5.2.0 requires Python 3.10 or newer. - Transformers 5.1.0 requires PyTorch 2.4 or newer; 4.56.0 had already raised the floor to PyTorch 2.2. - TensorFlow and JAX backends are deprecated since 4.53.0. ### Migrate tokenizer code for v5 - Call the tokenizer instead of `encode_plus`. - `decode` accepts single and batched inputs; `batch_decode` is no longer required for the batched case. - `apply_chat_template` returns `BatchEncoding`; select `input_ids` rather than treating the result as a tensor or list. - Use `text_target` instead of `as_target_tokenizer`; use `word_ids()` instead of `BatchEncoding.words()`. - A tokenizer constructor does not accept `vocab_file`; use `from_pretrained` for file-backed loading, or construct with `vocab` and `merges`. - Repositories must declare `model_type`; `AutoTokenizer` no longer infers it from a directory name as of 5.2.0. ### Migrate model loading - Prefer `dtype`; `torch_dtype` is transitional. In v5, `from_pretrained` defaults `dtype="auto"`, preserving the checkpoint dtype. - Replace `use_auth_token` with `token`. - Replace top-level `load_in_4bit` and `load_in_8bit` with a `quantization_config`, such as `BitsAndBytesConfig`. - Pass configuration values by keyword: configuration classes are dataclasses and reject positional arguments as of 5.4.0. - Use a local directory or Hub repository for configs; arbitrary config URLs are not supported in v5. ### Migrate configuration access - Read rotary settings from `config.rope_parameters`, not direct attributes such as `config.rope_theta`. - Read architecture-specific values from subconfigs, for example `config.text_config.vocab_size` for Qwen-VL. - Use `config.backbone_config` as the source of truth for backbone models. - Non-generative configs do not have `generation_config`. - Preserve heterogeneous `per_layer_config` data instead of assuming one global attention configuration. ### Remove retired APIs and strategies - `transformers.agents` is removed; migrate agent code to `smolagents`. - DoLa, Contrastive Search, Group Beam Search, and Constrained Beam Search are no longer built in. The first two are available as trusted custom-generation implementations. - Head masking, head pruning, BERT-like relative positional biases, `torchscript`, and `torch.fx` integrations are removed in v5. - `pad_to_max_length`, `EncoderDecoderCache.batch_split`, the ASR pipeline's `num_frames`, and the misspelled `AnnotionFormat` are removed. - The Apex integration is removed; use native PyTorch mixed precision and fused operations. ### Update custom model and attention code - Custom attention implementations must adopt the 5.2.0 attention-mask interface and call rotary functions directly rather than through `self.rotary_fn` as of 5.6.0. - Generation now supplies full `input_ids` to `prepare_inputs_for_generation`; do not slice inputs with `cache_position`. - Most direct model `forward` methods no longer accept `cache_position` as of 5.4.0; let `generate` manage it. - Inputs use the plural name `inputs_embeds`. - Custom integrations must migrate away from private layer, mask, cache, hybrid-attention, linear-layer, and multimodal processor helpers. ## Loading and execution quick reference ### Make kernel selection explicit - Installing `kernels` does not activate decorated forward methods. Pass `use_kernels=True` or choose a registered `attn_implementation`. - Linear-attention families use native fallbacks by default as of 5.15.1; pass `use_kernels=True` to retain kernel-backed execution. - ModernBERT no longer chooses Flash Attention implicitly as of 5.2.0. - T5-family models can select SDPA or another registered backend; request `attn_implementation="eager"` when eager execution is required. - Unsupported `output_attentions=True` combinations fail instead of silently falling back to eager attention. ### Respect quantization constraints - Tensor-parallel quantized inference in 4.52.1 supports only `compressed-tensors`, `fp8`, and `fp8-fbgemm`. - FP-Quant initially implements post-training MXFP4; accelerated execution needs Blackwell hardware and QuTLASS, while `pseudoquant=True` emulates it. - Quantizing an already quantized model is an error as of 4.56.0. - torchao requires version 0.15.0 or newer as of 5.4.0. - Use the full loading and kernel matrix in the loading reference before combining quantization, device maps, tensor parallelism, or custom kernels. ## Generation and serving quick reference ### Use current cache contracts - Cache implementations initialize caches explicitly and return `Cache` objects; use `past_key_values`, not `past_key_value`. - Sliding-window generation enforces configured limits and retains only needed state. Output can differ from older effectively unbounded behavior. - Crop a cache by negative relative offset, for example `cache.crop(-tokens_to_remove)`; absolute target lengths are unsupported as of 5.15.1. - Native caches replace custom Mamba and mixed Mamba-attention workarounds. ### Choose the right generation path - Assisted generation accepts an assistant from another architecture and also works with sampling. - `custom_generate` executes code. Both Hub and local implementations require explicit `trust_remote_code=True` where specified. - Continuous batching uses `generate_batch`; it supports paged attention, sliding-window models, CPU offload, tensor parallelism, and request controls. - Repetition penalties require `input_ids` as of 5.9.0. ### Serve locally - `transformers serve` is intended for experimentation and private local use. It exposes chat, responses, transcription, model-listing, and legacy completions endpoints. - Requests naming a model other than the server's pinned model receive HTTP 400. Use `--compile` and `--model-timeout` where appropriate. - `transformers chat MODEL key=value` accepts `GenerationConfig`-style settings and can target the same local server. ## Processing and training quick reference ### Handle multimodal inputs deliberately - `apply_chat_template` accepts in-memory video, PIL images, audio/video chat content, and `image_url` entries where supported. - SAM3-family `text_embeds` expects full text embeddings, not pooler output. - Gemma 4 vision preprocessing uses fixed soft-token budgets and internal scaling; do not add ImageNet normalization. - The unified image-processor backend lives in `image_processing_utils`; the old fast module is removed. ### Check changed training semantics - `TrainingArguments.average_tokens_across_devices` defaults to enabled. - Final partial gradient-accumulation windows now receive correct loss scaling. - `Trainer` aligns model special tokens with the tokenizer and supports sequence parallel evaluation plus `ddp_static_graph`. - Corrected expert-parallel and FSDP behavior can change formerly wrong or NaN results; re-baseline affected training jobs. ## Working method 1. Inspect the installed Transformers, Python, PyTorch, accelerator, and quantization-package versions. 2. Locate the task in the reference index and apply only relevant guidance. 3. Treat explicit trust flags as security boundaries, especially for custom generation and previously remote-code model integrations. 4. Re-run representative preprocessing, generation, cached decoding, and training tests when a result-affecting correction applies. 5. Pin revisions for Hub kernels or Git-only model releases when reproducible artifacts matter.