Model fine-tuning standards
Criteria verified as of August 2026. Re-verify on the web before committing to anything (§8).
1. Scope and triggers
Applies to modifying the weights of someone else's pretrained model: the decision of whether to do it, the
method (full versus PEFT), the data, preference alignment, the mandatory
evaluation before and after, the licence of the starting weights and the deployment of the resulting
variant.
Triggers: peft, LoraConfig, get_peft_model, r=/lora_alpha/target_modules,
QLoRA, bitsandbytes 4-bit, trl, SFTTrainer, DPOTrainer, GRPOTrainer, KTOTrainer,
RLOOTrainer, RewardTrainer, unsloth, axolotl, adapter_model.safetensors,
adapter_config.json, merge_and_unload, chat template / chat_template, .jsonl of
conversations, "fine-tune the model with our documents", "make it learn our product",
"make it answer in our tone", "the model doesn't know X", "train on our tickets".
Domain thesis: fine-tuning is the last option, not the first. The order of increasing
cost and decreasing reversibility is:
- Prompt (instructions, examples, structured output). Marginal cost, change in
seconds, reversible. Solves format, tone, judgement and bounded tasks.
- Retrieval (RAG). Solves knowledge: your own facts, updatable, citable and
deletable. It is what people believe fine-tuning solves.
- Fine-tuning. Solves behaviour: rigid format, consistent style, narrow
repetitive task, reduced prompt length and lower latency/cost per request.
- Training your own (
deep-learning-standards). Another order of magnitude of cost.
What fine-tuning fixes and what it does not:
| It does fix |
It does not fix |
| Stable output format without long instructions |
Not knowing facts — the most common and most expensive mistake |
| Consistent style, tone and register |
Knowledge that changes (prices, catalogue, policies) |
| Narrow, repetitive task (classify, extract, rewrite) |
Traceability and source citation |
| Consistency across runs and less instruction drift |
Deleting a datum at the subject's request |
| Shorter prompt → fewer tokens, less latency |
General reasoning the base model does not have |
| Meeting your own schema or taxonomy |
Making the model "not hallucinate" |
Hard rule: if the answer to "what do you expect it to learn?" is a set of facts, the
solution is retrieval, not fine-tuning. Putting facts into the weights is expensive, not verifiable,
not updatable and not deletable.
Corollary: fine-tuning is not training. You start from someone else's weights, move them a little and the
result inherits the capabilities and the legal restrictions of the base model (§5).
Not applicable:
llm-app-engineering-standards and rag-standards (written — critical boundary):
building a product on a third-party LLM is theirs, as is retrieval —
ingestion, chunking, embeddings, index, hybrid, reranking, citation. Here, modifying the
weights. The order prompt → retrieval → fine-tuning crosses all three skills: from this side
only the third rung is asserted and the burden of proof of having exhausted the previous two
falls here. If the prompt or the index solve the case, this skill declares there is no
project.
mlops-standards (written — critical boundary): the model's lifecycle in
production is theirs — artifact registry and versioning, promotion, deployment, rollback
to the previous weights, drift, retraining and retirement. Here, how the model is tuned and evaluated
before it gets there. An adapter you trained is a model artifact of your own
and enters their registry with its card and its lineage.
llm-evaluation-standards (written): measuring non-deterministic systems is theirs —
evaluation set, LLM judges, significance, CI gates, benchmark contamination.
Here evaluation before and after is required (§6) and what to measure is defined (target task
and general capabilities); the how, there.
local-inference-standards (written): serving your own weights is theirs — vLLM, llama.cpp,
batching, KV cache, hot adapters, serving quantisation. Here, the decision to
merge or serve adapters and its maintenance cost; the engine, there.
deep-learning-standards (this same wave): training your own network. Fine-tuning is not
training, but it inherits its mechanics: mixed precision, gradient accumulation, checkpoints
and seeds are governed there.
classical-ml-standards (this same wave): with thousands of labelled examples, a
classical classifier over embeddings usually wins on cost, latency and debuggability over
fine-tuning a generative model. It is the forgotten baseline.
gpu-computing-standards (the GPU as a resource that is provisioned, shared and paid for);
mlsecops-standards (weight provenance, safe formats and attacks on the model, including
corpus poisoning and backdoors); ai-governance-standards (risk, the AI
Act and inventory — careful: tuning and publishing can turn you into a provider for
regulatory purposes); opensource-licensing-standards (the licence policy; here only the
specific restriction of a given set of weights); privacy-engineering-standards (personal data and
memorisation); data-engineering-standards, data-governance-quality-standards;
finops-standards, green-it-standards; python-standards; computer-vision-standards,
nlp-standards, multimodal-genai-standards.
2. Default decisions
Verify the latest version on the web before pinning it in a real project (§8).
| Piece |
Choice |
Verified version (Aug 2026, PyPI) |
Licence |
| PEFT / adapters |
peft |
0.20.0 (28 Jul 2026) |
Apache-2.0 (repo header) |
| SFT and preference trainers |
trl |
1.9.2 (28 Jul 2026) |
Apache-2.0 (PyPI metadata) |
| Models and tokenizers |
transformers |
5.14.1 (16 Jul 2026) |
Apache-2.0 |
| Distribution / launching |
accelerate |
1.14.0 (11 Jun 2026) |
Apache-2.0 |
| Training quantisation (QLoRA) |
bitsandbytes |
0.50.0 (24 Jul 2026) |
MIT (PyPI metadata) |
| Data |
datasets |
5.0.1 (28 Jul 2026) |
Apache-2.0 |
| Weight format |
safetensors |
0.8.0 (9 Jun 2026) |
verify (§8) |
Method: LoRA/QLoRA by default; full fine-tuning only if LoRA with a sufficient rank has already
fallen short and it is measured. QLoRA (4-bit quantised base + adapter) when VRAM
rules; assume a quality penalty that must be measured, not assumed.
Variants: peft today documents LoRA with its own variants (DoRA, BD-LoRA, KaSA,
MonteCLoRA, VeLoRA in its documentation index) as well as different families (AdaLoRA, LoHa,
LoKr, OFT/BOFT, IA3, prompt/prefix tuning and others). Criteria: plain LoRA is the starting
point; a variant enters only with your own measurement justifying it, not for being new.
3. Hyperparameters that really matter
Four decisions concentrate the result: target layers, rank, learning rate and effective batch
size. Citable reference: the "LoRA Without Regret" guide in the TRL
documentation, which reproduces findings from a Thinking Machines Lab paper (Schulman et al., 2025).
Verbatim from that guide:
- "The authors recommend applying LoRA to all weight matrices rather than limiting it to
attention layers, as increasing the rank does not compensate for this restriction." →
target_modules="all-linear" as a starting point, not just q_proj/v_proj.
- "For datasets that exceed LoRA capacity, LoRA underperforms FullFT" → the rank must cover
the capacity the dataset demands. The guide's table recommends rank 256 for SFT at
"post-training scale" and 1-32 for RL.
- "Counterintuitively, the blog post recommends using a higher learning rate than for full
fine-tuning"; in its table, 1.0e-5 for LoRA against 1.0e-6 for full fine-tuning, and "The
1/r scaling in LoRA makes the optimal learning rate approximately rank-independent".
- "In some scenarios, LoRA is less tolerant of large batch sizes than full fine-tuning." → the
guide recommends an effective batch < 32.
Declared methodology and limit: results from a lab blog post
reproduced by HuggingFace on specific models and datasets (SmolLM3-3B, Llama-3.2-1B/3.1-8B,
tulu-3-sft-mixture, OpenThoughts-114k, OpenR1-Math-220k); it is not peer-reviewed literature and
does not generalise to your model without measuring it. Its compute claim ("~67% of the compute")
is not adopted here as a figure: it comes with no hardware or conditions (§8). lora_alpha is a scale
relative to the rank: it is set together with it and not touched blindly.
4. Preference alignment
Correct order: SFT first (format and task), preferences after (choosing between
valid answers). Skipping SFT to go straight to preferences is the usual sequencing
error.
Real status verified in the trl documentation (docs index, Aug 2026): the
stable trainers are DPO, GRPO, KTO, Reward, RLOO and SFT. In the
experimental section live, among others, PPO, ORPO, CPO, BCO, Online DPO, Nash-MD, GSPO-token,
SDPO, A2PO, GMPO, GOLD, GKD, PRM and distillation.
Reading of the criteria: DPO is neither superseded nor withdrawn — it remains a first-class
trainer, and alongside it reinforcement methods with verifiable reward (GRPO,
RLOO) have consolidated, the real novelty of the last cycle; the proliferation of acronyms lives in experimental for
a reason. Criteria: DPO for pairwise preferences; GRPO/RLOO when there is a verifiable reward
(code that compiles, a test that passes, a correct numerical result); everything else, only with your own
measurement.
When it is worth it: you already have decent SFT, there is a consistent preference you cannot
write down as an instruction and you have hundreds or thousands of reliable comparisons. With less, you are
injecting noise and risking degradation.
5. Data and licences — the part that decides the result
5.1 The training set is 90 % of the result
- Quality over quantity. Hundreds or a few thousand excellent and consistent examples
beat tens of thousands of mediocre ones; a contradictory criterion repeated teaches the contradiction.
- Format consistency: the data is the specification and any deviation is learned.
- Conversation format: use the base model's chat template (special tokens,
roles, end markers). A mismatch between the training template and the inference one is
the number one cause of "the fine-tuned model answers oddly", and it is train/serve skew.
- Edges and negatives: include examples where the correct thing is to refuse, ask for clarification or
say it does not know. If you do not put them in, the model learns to make things up.
- Contamination with the eval: deduplicate the corpus against the evaluation set before
training, by exact match and by similarity. A contaminated eval measures nothing and cannot be
decontaminated after the fact: you have to retrain.
- Synthetic data: useful for volume and edges; it inherits bias and errors from the generator,
collapses diversity, and generating with a third-party model to train yours may
violate their terms of use (check the provider's terms, not just the licence of the
weights). Label the synthetic origin and review a sample by hand.
- PII: lawfulness, minimisation and retention before building the corpus
(
privacy-engineering-standards). The model memorises: there is no DELETE in a set of weights.
5.2 Weight licences: they are not software licences
Central trap of the domain: the licence of a set of weights is not a software licence and
"open source" applied to a model almost never means what it seems. Cases read in their text:
| Weights |
Licence |
Real restriction (verbatim from the text) |
| Llama 3.3 (text read raw; Llama 4 has its own agreement, §8) |
Llama Community License Agreement (not OSI) |
Mandatory attribution: "prominently display "Built with Llama""; forced renaming: "you shall also include "Llama" at the beginning of any such AI model name" if you train another model with its materials or outputs; user limit: "is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta"; acceptable use policy incorporated by reference |
| Gemma |
Gemma Terms of Use (not OSI) |
"You must not use any of the Gemma Services: for the restricted uses set forth in the Gemma Prohibited Use Policy"; mandatory propagation: "You must provide all third party recipients of Gemma or Model Derivatives a copy of this Agreement" and "include the use restrictions referenced in Section 3.2 as an enforceable provision"; "Google reserves the right to restrict (remotely or otherwise) usage of any of the Gemma Services that Google reasonably believes are in violation of this Agreement." |
| DeepSeek-V3 |
DeepSeek License Agreement v1.0 (not OSI) |
Use restrictions in its Attachment A, among them "For military use in any way" and "For fully automated decision making that adversely impacts an individual's legal rights"; mandatory propagation to derivatives; PRC jurisdiction and law |
| DeepSeek-R1 |
MIT (repo LICENSE) |
Genuinely permissive — same provider family, different licence: it is verified per model, never per brand |
| Qwen3 (e.g. Qwen3-8B) |
Apache-2.0 (repo LICENSE) |
Permissive; keep the notice and NOTICE |
| Mistral |
Varies per model (e.g. license: apache-2.0 on the Mistral-Small-3.2-24B-Instruct-2506 card) |
Other models from the same provider are published under a research licence: verify the specific model |
Rules that follow:
- None of the licences in the "not OSI" column is free software or open source: field-of-use
restrictions are incompatible with the open source definition by
construction. Calling them "open source" in internal or commercial documentation is a legal risk
of its own.
- The code repository's licence is not the weights' licence. Verified example:
the
google-deepmind/gemma repository publishes its LICENSE under Apache-2.0, while the model
card declares license: gemma and points to the Gemma Terms of Use. You read the
document that accompanies the weights, not the code's.
- Restrictions are inherited and propagated: your fine-tuned variant is subject to the
base's terms and, in several of these cases, you are obliged to impose them on whoever receives it.
- The corpus licence also restricts: a dataset with a non-commercial or research-only
clause contaminates the use of the resulting model. Data and weight licences are recorded
in the model card.
- The general licence policy, the CI gate and the SBOM/AIBOM belong to
opensource-licensing-standards.
6. Mandatory evaluation, before and after
- Prompt baseline mandatory. Without the base model's metric with the best
reasonable prompt —and, if applicable, with retrieval— you do not know whether the tuning added anything. It is the
entry gate to the project, not a later report.
- Two measurement axes, always: (1) target task, with your own uncontaminated eval;
(2) general capabilities — catastrophic forgetting: tuning narrows the model, so measure
before and after capabilities that are not the task (instruction following,
reasoning, language, refusals). An improvement on the task that breaks the language or safety is
not an improvement.
- Safety behaviour: tuning can degrade the base's refusals even if the
data is not malicious; it is measured before publishing (
mlsecops-standards for red teaming).
- Methodology:
llm-evaluation-standards. Here only what is measured and when it
blocks is fixed: if the task does not improve significantly over the prompt baseline, or if
general capabilities drop above the declared tolerance, the artifact is not
promoted.
- Experiment logging (data, hyperparameters, seed, exact base by revision/digest) and
artifact promotion:
mlops-standards.
7. Deployment, sustainability and prohibitions
- Merge or serve adapters: merging (
merge_and_unload) gives a single artifact, simple
to serve and quantise, losing modularity and multiplying storage per variant;
serving adapters over a shared base allows many variants with a single copy of the
weights, at the cost of complexity and compatibility ties in the engine. Rule: one variant
→ merge; several over the same base → adapters. The engine: local-inference-standards.
- The real cost of a family of variants is maintenance, not training: each
one needs an eval, a card, retraining when the base changes and retirement. Before the
second one, ask whether routing by prompt is enough.
- When a better version of the base model comes out, the work may become obsolete: fine-tuning
is perishable and tied to a specific base. Plan with that expiry in mind and keep the
dataset, which is the asset that survives.
FORBIDDEN:
- ❌ Fine-tuning to add factual knowledge. That is retrieval (
rag-standards).
- ❌ Fine-tuning without a prompt baseline (and, where applicable, without a retrieval baseline).
- ❌ Training on data contaminated with the evaluation set.
- ❌ Publishing a fine-tuned model without declaring the base model's licence, its inherited
restrictions and the corpus licence.
- ❌ Calling weights with use restrictions "open source".
- ❌ Inferring the licence of a set of weights from the code repository's, from the provider's
brand or from a third-party summary: you read the text that accompanies the model.
- ❌ Publishing without measuring catastrophic forgetting of general capabilities and safety
behaviour.
- ❌ Training with a chat template different from the one you will use at inference.
- ❌ Generating synthetic data with a provider whose terms forbid training competing
models, without having read those terms.
- ❌ Skipping SFT and applying preference methods directly on the base model.
- ❌ Maintaining a family of variants without an automated eval per variant.
- ❌ Adopting a "new" LoRA variant or preference method without your own measurement.
8. Mandatory web verification
Before pinning anything in a real project, check on the web:
- Latest stable releases and breaking changes of
transformers, peft, trl, accelerate,
datasets and bitsandbytes: trl moves trainers between stable and experimental across
versions, and transformers 5.x already introduced API breakage. Read the repo's documentation
index, not a tutorial.
- Status of DPO against more recent methods: verify in the official
trl
documentation which trainers are in Trainers (stable) and which in Experimental. What is written
here comes from the repository's _toctree.yml in Aug 2026 and expires fast.
- The licence of EVERY weights model you use, read in its full text, including the
acceptable use policies incorporated by reference and the provider's terms if you use
their API to generate data. Declared gaps: (a) the text of the Llama Community License
cited here corresponds to Llama 3.3 (version date 6 Dec 2024) read raw from the
meta-llama/llama-models repository; Llama 4 has its own agreement (effective date
5 Apr 2025) which has not been read in full in this write-up — do not assume the clauses
match; (b) the Gemma Terms of Use clauses were obtained by extraction from the
official page, not from a raw file: re-read them at ai.google.dev/gemma/terms
before basing a legal decision on them; (c) the safetensors licence was not verified.
- OSI approval: none of the weight licences with use restrictions cited appears
as OSI-approved, and by construction it cannot be (field-of-use restrictions
are incompatible with the definition). Verify the current list and the status of the Open Source
AI Definition (OSAID) before using the "open source" label in documentation.
- Declared gaps: (a) no figure for quality, compute saving or memory of LoRA/QLoRA
against full fine-tuning — the available ones come from lab blogs or reproductions
on specific models, with no homogeneous conditions; (b) no minimum dataset size — the
ranges that circulate ("a thousand examples are enough") come with no task, model or metric: the number comes
out of your evaluation curve as you vary the size.
- CVEs and maintenance status of
bitsandbytes and the quantisation kernels, and
provenance/signature of the base weights (mlsecops-standards).
If the web contradicts this document, the web wins — flag the discrepancy.
1---2name: model-finetuning-standards3description: Fine-tuning an existing model's weights as the last resort, after prompting and retrieval. Use when deciding between prompt engineering, RAG and training, running full fine-tuning versus PEFT with peft LoraConfig (r, lora_alpha, target_modules="all-linear", lora_dropout), QLoRA with bitsandbytes 4-bit, or DoRA and other LoRA variants, training with trl SFTTrainer, DPOTrainer, GRPOTrainer, KTOTrainer, RLOOTrainer or RewardTrainer, Unsloth or Axolotl configs, building a chat-formatted instruction dataset with chat templates and JSONL conversations, checking eval-set contamination and synthetic-data provenance, measuring catastrophic forgetting of general capability before and after, reading the weights licence of Llama, Gemma, Qwen, Mistral or DeepSeek checkpoints and whether it is OSI-approved, merging adapters with merge_and_unload versus serving adapters at runtime, or maintaining a family of fine-tuned variants.4metadata:5 source: https://skillmd.com/skills/serialexperimentslainnnn/model-finetuning-standards6---78# Model fine-tuning standards910Criteria verified as of **August 2026**. Re-verify on the web before committing to anything (§8).1112## 1. Scope and triggers1314Applies to **modifying the weights of someone else's pretrained model**: the decision of whether to do it, the15method (full versus PEFT), the data, preference alignment, the mandatory16evaluation before and after, the licence of the starting weights and the deployment of the resulting17variant.1819Triggers: `peft`, `LoraConfig`, `get_peft_model`, `r=`/`lora_alpha`/`target_modules`,20`QLoRA`, `bitsandbytes` 4-bit, `trl`, `SFTTrainer`, `DPOTrainer`, `GRPOTrainer`, `KTOTrainer`,21`RLOOTrainer`, `RewardTrainer`, `unsloth`, `axolotl`, `adapter_model.safetensors`,22`adapter_config.json`, `merge_and_unload`, chat template / `chat_template`, `.jsonl` of23conversations, "fine-tune the model with our documents", "make it learn our product",24"make it answer in our tone", "the model doesn't know X", "train on our tickets".2526**Domain thesis**: **fine-tuning is the last option, not the first.** The order of increasing27cost and decreasing reversibility is:28291. **Prompt** (instructions, examples, structured output). Marginal cost, change in30 seconds, reversible. Solves format, tone, judgement and bounded tasks.312. **Retrieval (RAG)**. Solves **knowledge**: your own facts, updatable, citable and32 deletable. It is what people believe fine-tuning solves.333. **Fine-tuning**. Solves **behaviour**: rigid format, consistent style, narrow34 repetitive task, reduced prompt length and lower latency/cost per request.354. **Training your own** (`deep-learning-standards`). Another order of magnitude of cost.3637**What fine-tuning fixes and what it does not**:3839| It does fix | It does not fix |40|---|---|41| Stable output format without long instructions | **Not knowing facts** — the most common and most expensive mistake |42| Consistent style, tone and register | Knowledge that changes (prices, catalogue, policies) |43| Narrow, repetitive task (classify, extract, rewrite) | Traceability and source citation |44| Consistency across runs and less instruction drift | Deleting a datum at the subject's request |45| Shorter prompt → fewer tokens, less latency | General reasoning the base model does not have |46| Meeting your own schema or taxonomy | Making the model "not hallucinate" |4748**Hard rule**: if the answer to "what do you expect it to learn?" is a set of facts, the49solution is retrieval, not fine-tuning. Putting facts into the weights is expensive, not verifiable,50not updatable and not deletable.5152Corollary: **fine-tuning is not training.** You start from someone else's weights, move them a little and the53result inherits the capabilities **and the legal restrictions** of the base model (§5).5455**Not applicable**:5657- `llm-app-engineering-standards` and `rag-standards` (**written — critical boundary**):58 **building a product on a third-party LLM is theirs**, as is **retrieval** —59 ingestion, *chunking*, embeddings, index, hybrid, *reranking*, citation. **Here, modifying the60 weights.** The order prompt → retrieval → fine-tuning crosses all three skills: from this side61 only the third rung is asserted and **the burden of proof of having exhausted the previous two62 falls here**. If the prompt or the index solve the case, this skill declares there is no63 project.64- `mlops-standards` (**written — critical boundary**): **the model's lifecycle in65 production is theirs** — artifact registry and versioning, promotion, deployment, *rollback*66 to the previous weights, drift, retraining and retirement. **Here, how the model is tuned and evaluated67 before it gets there.** An adapter you trained **is a model artifact of your own**68 and enters their registry with its card and its lineage.69- `llm-evaluation-standards` (**written**): **measuring non-deterministic systems is theirs** —70 evaluation set, LLM judges, significance, CI gates, *benchmark* contamination.71 Here evaluation before and after is **required** (§6) and **what** to measure is defined (target task72 **and** general capabilities); the **how**, there.73- `local-inference-standards` (**written**): **serving your own weights is theirs** — vLLM, llama.cpp,74 batching, KV cache, hot adapters, serving quantisation. Here, the decision to75 **merge or serve adapters** and its maintenance cost; the engine, there.76- `deep-learning-standards` (**this same wave**): training your own network. **Fine-tuning is not77 training**, but it inherits its mechanics: mixed precision, gradient accumulation, *checkpoints*78 and seeds are governed there.79- `classical-ml-standards` (**this same wave**): with thousands of labelled examples, **a80 classical classifier over embeddings usually wins on cost, latency and debuggability** over81 fine-tuning a generative model. It is the forgotten baseline.82- `gpu-computing-standards` (**the GPU as a resource that is provisioned, shared and paid for**);83 `mlsecops-standards` (**weight provenance, safe formats and attacks on the model**, including84 corpus poisoning and backdoors); `ai-governance-standards` (**risk, the AI85 Act and inventory** — careful: **tuning and publishing can turn you into a provider** for86 regulatory purposes); `opensource-licensing-standards` (**the licence policy**; here only the87 specific restriction of a given set of weights); `privacy-engineering-standards` (personal data and88 memorisation); `data-engineering-standards`, `data-governance-quality-standards`;89 `finops-standards`, `green-it-standards`; `python-standards`; `computer-vision-standards`,90 `nlp-standards`, `multimodal-genai-standards`.9192## 2. Default decisions9394> Verify the latest version on the web before pinning it in a real project (§8).9596| Piece | Choice | Verified version (Aug 2026, PyPI) | Licence |97|---|---|---|---|98| PEFT / adapters | `peft` | 0.20.0 (28 Jul 2026) | Apache-2.0 (repo header) |99| SFT and preference trainers | `trl` | 1.9.2 (28 Jul 2026) | Apache-2.0 (PyPI metadata) |100| Models and tokenizers | `transformers` | 5.14.1 (16 Jul 2026) | Apache-2.0 |101| Distribution / launching | `accelerate` | 1.14.0 (11 Jun 2026) | Apache-2.0 |102| Training quantisation (QLoRA) | `bitsandbytes` | 0.50.0 (24 Jul 2026) | MIT (PyPI metadata) |103| Data | `datasets` | 5.0.1 (28 Jul 2026) | Apache-2.0 |104| Weight format | `safetensors` | 0.8.0 (9 Jun 2026) | verify (§8) |105106**Method**: **LoRA/QLoRA by default**; full fine-tuning only if LoRA with a sufficient rank has already107fallen short **and it is measured**. QLoRA (4-bit quantised base + adapter) when VRAM108rules; assume a quality penalty that must be measured, not assumed.109110**Variants**: `peft` today documents LoRA with its own variants (**DoRA, BD-LoRA, KaSA,111MonteCLoRA, VeLoRA** in its documentation index) as well as different families (AdaLoRA, LoHa,112LoKr, OFT/BOFT, IA3, prompt/prefix tuning and others). **Criteria: plain LoRA is the starting113point; a variant enters only with your own measurement justifying it**, not for being new.114115## 3. Hyperparameters that really matter116117Four decisions concentrate the result: **target layers, rank, learning rate and effective batch118size**. Citable reference: the *"LoRA Without Regret"* guide in the TRL119documentation, which reproduces findings from a Thinking Machines Lab paper (Schulman et al., 2025).120Verbatim from that guide:121122- *"The authors recommend applying LoRA to all weight matrices rather than limiting it to123 attention layers, as increasing the rank does not compensate for this restriction."* →124 `target_modules="all-linear"` as a starting point, not just `q_proj`/`v_proj`.125- *"For datasets that exceed LoRA capacity, LoRA underperforms FullFT"* → the rank must cover126 the capacity the dataset demands. The guide's table recommends rank **256** for SFT at127 "post-training scale" and **1-32** for RL.128- *"Counterintuitively, the blog post recommends using a higher learning rate than for full129 fine-tuning"*; in its table, 1.0e-5 for LoRA against 1.0e-6 for full fine-tuning, and *"The130 1/r scaling in LoRA makes the optimal learning rate approximately rank-independent"*.131- *"In some scenarios, LoRA is less tolerant of large batch sizes than full fine-tuning."* → the132 guide recommends an effective batch < 32.133134**Declared methodology and limit**: results from a lab blog post135reproduced by HuggingFace on specific models and datasets (SmolLM3-3B, Llama-3.2-1B/3.1-8B,136tulu-3-sft-mixture, OpenThoughts-114k, OpenR1-Math-220k); **it is not peer-reviewed literature and137does not generalise to your model without measuring it**. Its compute claim ("~67% of the compute")138**is not adopted here as a figure**: it comes with no hardware or conditions (§8). `lora_alpha` is a scale139relative to the rank: it is set together with it and not touched blindly.140141## 4. Preference alignment142143Correct order: **SFT first** (format and task), **preferences after** (choosing between144valid answers). Skipping SFT to go straight to preferences is the usual sequencing145error.146147**Real status verified in the `trl` documentation (docs index, Aug 2026)**: the148**stable** trainers are **DPO, GRPO, KTO, Reward, RLOO and SFT**. In the149**experimental** section live, among others, **PPO, ORPO, CPO, BCO, Online DPO, Nash-MD, GSPO-token,150SDPO, A2PO, GMPO, GOLD, GKD, PRM and distillation**.151152Reading of the criteria: **DPO is neither superseded nor withdrawn — it remains a first-class153trainer**, and alongside it reinforcement methods with verifiable reward (GRPO,154RLOO) have consolidated, the real novelty of the last cycle; the proliferation of acronyms lives in experimental for155a reason. **Criteria: DPO for pairwise preferences; GRPO/RLOO when there is a verifiable reward156(code that compiles, a test that passes, a correct numerical result); everything else, only with your own157measurement.**158159**When it is worth it**: you already have decent SFT, there is a consistent preference you cannot160write down as an instruction and you have hundreds or thousands of reliable comparisons. With less, you are161injecting noise and risking degradation.162163## 5. Data and licences — the part that decides the result164165### 5.1 The training set is 90 % of the result166167- **Quality over quantity.** Hundreds or a few thousand **excellent and consistent** examples168 beat tens of thousands of mediocre ones; a contradictory criterion repeated teaches the contradiction.169- **Format consistency**: the data is the specification and any deviation is learned.170- **Conversation format**: use the **base model's chat template** (special tokens,171 roles, end markers). A mismatch between the training template and the inference one is172 the number one cause of "the fine-tuned model answers oddly", and it is *train/serve skew*.173- **Edges and negatives**: include examples where the correct thing is to refuse, ask for clarification or174 say it does not know. If you do not put them in, the model learns to make things up.175- **Contamination with the eval**: deduplicate the corpus against the evaluation set **before**176 training, by exact match and by similarity. A contaminated eval measures nothing and cannot be177 decontaminated after the fact: you have to retrain.178- **Synthetic data**: useful for volume and edges; it inherits bias and errors from the generator,179 collapses diversity, and **generating with a third-party model to train yours may180 violate their terms of use** (check the provider's terms, not just the licence of the181 weights). Label the synthetic origin and review a sample by hand.182- **PII**: lawfulness, minimisation and retention before building the corpus183 (`privacy-engineering-standards`). The model memorises: **there is no `DELETE` in a set of weights**.184185### 5.2 Weight licences: they are not software licences186187**Central trap of the domain**: the licence of a set of weights is **not** a software licence and188"open source" applied to a model almost never means what it seems. Cases read in their text:189190| Weights | Licence | Real restriction (verbatim from the text) |191|---|---|---|192| Llama 3.3 (text read raw; **Llama 4 has its own agreement**, §8) | *Llama Community License Agreement* (not OSI) | Mandatory attribution: *"prominently display "Built with Llama""*; forced renaming: *"you shall also include "Llama" at the beginning of any such AI model name"* if you train another model with its materials or outputs; **user limit**: *"is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta"*; acceptable use policy incorporated by reference |193| Gemma | *Gemma Terms of Use* (not OSI) | *"You must not use any of the Gemma Services: for the restricted uses set forth in the Gemma Prohibited Use Policy"*; mandatory propagation: *"You must provide all third party recipients of Gemma or Model Derivatives a copy of this Agreement"* and *"include the use restrictions referenced in Section 3.2 as an enforceable provision"*; *"Google reserves the right to restrict (remotely or otherwise) usage of any of the Gemma Services that Google reasonably believes are in violation of this Agreement."* |194| DeepSeek-V3 | *DeepSeek License Agreement v1.0* (not OSI) | Use restrictions in its Attachment A, among them *"For military use in any way"* and *"For fully automated decision making that adversely impacts an individual's legal rights"*; mandatory propagation to derivatives; PRC jurisdiction and law |195| DeepSeek-R1 | **MIT** (repo `LICENSE`) | Genuinely permissive — **same provider family, different licence**: it is verified per model, never per brand |196| Qwen3 (e.g. Qwen3-8B) | **Apache-2.0** (repo `LICENSE`) | Permissive; keep the notice and NOTICE |197| Mistral | **Varies per model** (e.g. `license: apache-2.0` on the Mistral-Small-3.2-24B-Instruct-2506 card) | Other models from the same provider are published under a research licence: **verify the specific model** |198199Rules that follow:200201- **None of the licences in the "not OSI" column is free software or open source**: field-of-use202 restrictions are incompatible with the open source definition by203 construction. Calling them "open source" in internal or commercial documentation is a legal risk204 of its own.205- **The code repository's licence is not the weights' licence.** Verified example:206 the `google-deepmind/gemma` repository publishes its `LICENSE` under Apache-2.0, while the model207 card declares `license: gemma` and points to the *Gemma Terms of Use*. **You read the208 document that accompanies the weights, not the code's.**209- **Restrictions are inherited and propagated**: your fine-tuned variant is subject to the210 base's terms and, in several of these cases, you are obliged to impose them on whoever receives it.211- **The corpus licence also restricts**: a dataset with a non-commercial or research-only212 clause contaminates the use of the resulting model. Data and weight licences are recorded213 in the model card.214- The general licence policy, the CI gate and the SBOM/AIBOM belong to215 `opensource-licensing-standards`.216217## 6. Mandatory evaluation, before and after218219- **Prompt baseline mandatory.** Without the base model's metric with the best220 reasonable prompt —and, if applicable, with retrieval— you do not know whether the tuning added anything. **It is the221 entry gate to the project, not a later report.**222- **Two measurement axes, always**: (1) **target task**, with your own uncontaminated eval;223 (2) **general capabilities — catastrophic forgetting**: tuning narrows the model, so measure224 before and after capabilities that are **not** the task (instruction following,225 reasoning, language, refusals). An improvement on the task that breaks the language or safety is226 not an improvement.227- **Safety behaviour**: tuning can degrade the base's refusals even if the228 data is not malicious; it is measured before publishing (`mlsecops-standards` for *red teaming*).229- **Methodology**: `llm-evaluation-standards`. Here only **what** is measured and **when** it230 blocks is fixed: if the task does not improve significantly over the prompt baseline, or if231 general capabilities drop above the declared tolerance, **the artifact is not232 promoted**.233- Experiment logging (data, hyperparameters, seed, exact base by revision/digest) and234 artifact promotion: `mlops-standards`.235236## 7. Deployment, sustainability and prohibitions237238- **Merge or serve adapters**: merging (`merge_and_unload`) gives a single artifact, simple239 to serve and quantise, losing modularity and multiplying storage per variant;240 serving adapters over a shared base allows many variants with a single copy of the241 weights, at the cost of complexity and compatibility ties in the engine. Rule: **one variant242 → merge; several over the same base → adapters.** The engine: `local-inference-standards`.243- **The real cost of a family of variants is maintenance, not training**: each244 one needs an eval, a card, retraining when the base changes and retirement. Before the245 second one, ask whether routing by prompt is enough.246- **When a better version of the base model comes out, the work may become obsolete**: fine-tuning247 is perishable and tied to a specific base. Plan with that expiry in mind and keep **the248 dataset**, which is the asset that survives.249250**FORBIDDEN**:251252- ❌ **Fine-tuning to add factual knowledge**. That is retrieval (`rag-standards`).253- ❌ **Fine-tuning without a prompt baseline** (and, where applicable, without a retrieval baseline).254- ❌ **Training on data contaminated with the evaluation set**.255- ❌ **Publishing a fine-tuned model without declaring the base model's licence**, its inherited256 restrictions and the corpus licence.257- ❌ Calling weights with use restrictions "open source".258- ❌ Inferring the licence of a set of weights from the code repository's, from the provider's259 brand or from a third-party summary: you read the text that accompanies the model.260- ❌ Publishing without measuring **catastrophic forgetting** of general capabilities and safety261 behaviour.262- ❌ Training with a chat template different from the one you will use at inference.263- ❌ Generating synthetic data with a provider whose terms forbid training competing264 models, without having read those terms.265- ❌ Skipping SFT and applying preference methods directly on the base model.266- ❌ Maintaining a family of variants without an automated eval per variant.267- ❌ Adopting a "new" LoRA variant or preference method without your own measurement.268269## 8. Mandatory web verification270271Before pinning anything in a real project, check on the web:272273- Latest stable releases and *breaking changes* of `transformers`, `peft`, `trl`, `accelerate`,274 `datasets` and `bitsandbytes`: **`trl` moves trainers between stable and experimental across275 versions**, and `transformers` 5.x already introduced API breakage. Read the repo's documentation276 index, not a tutorial.277- **Status of DPO against more recent methods**: verify in the official `trl`278 documentation which trainers are in `Trainers` (stable) and which in `Experimental`. What is written279 here comes from the repository's `_toctree.yml` in Aug 2026 and **expires fast**.280- **The licence of EVERY weights model you use, read in its full text**, including the281 acceptable use policies incorporated by reference and the provider's terms if you use282 their API to generate data. **Declared gaps**: (a) the text of the *Llama Community License*283 cited here corresponds to **Llama 3.3** (version date 6 Dec 2024) read raw from the284 `meta-llama/llama-models` repository; **Llama 4 has its own agreement** (effective date285 5 Apr 2025) which **has not been read in full in this write-up** — do not assume the clauses286 match; (b) the *Gemma Terms of Use* clauses were obtained by extraction from the287 official page, **not from a raw file**: re-read them at `ai.google.dev/gemma/terms`288 before basing a legal decision on them; (c) the `safetensors` licence was not verified.289- **OSI approval**: none of the weight licences with use restrictions cited appears290 as OSI-approved, and by construction it cannot be (field-of-use restrictions291 are incompatible with the definition). Verify the current list and the status of the *Open Source292 AI Definition* (OSAID) before using the "open source" label in documentation.293- **Declared gaps**: (a) no figure for quality, compute saving or memory of LoRA/QLoRA294 against full fine-tuning — the available ones come from lab blogs or reproductions295 on specific models, with no homogeneous conditions; (b) no minimum dataset size — the296 ranges that circulate ("a thousand examples are enough") come with no task, model or metric: the number comes297 out of your evaluation curve as you vary the size.298- CVEs and maintenance status of `bitsandbytes` and the quantisation *kernels*, and299 provenance/signature of the base weights (`mlsecops-standards`).300301If the web contradicts this document, **the web wins** — flag the discrepancy.