LLM Training Guide
A comprehensive guide for building and training large language models from scratch, based on the Manning book "Build a Large Language Model from Scratch".
Overview
This skill covers the complete LLM training pipeline:
- Tokenization - Converting text to token IDs
- Data Sampling - Preparing training data
- Token Embeddings - Vector representations
- Attention Mechanisms - Capturing word relationships
- LLM Architecture - Full model structure
- Pre-training - Training from scratch
- Fine-tuning - Adapting for specific tasks
Phase 1: Tokenization
Goal: Divide input text into tokens (IDs) in a meaningful way.
Key Concepts
- Tokens: The basic units the model processes (can be characters, words, or subwords)
- Vocabulary: The set of all unique tokens
- Token IDs: Numeric identifiers for each token in the vocabulary
Implementation Steps
- Build vocabulary from your training corpus
- Create token-to-ID mapping (tokenizer)
- Create ID-to-token mapping (for decoding)
- Encode text → convert to token IDs
- Decode IDs → convert back to text
Best Practices
- Use subword tokenization (like BPE or WordPiece) for better coverage
- Include special tokens:
<pad>, <unk>, <bos>, <eos>
- Keep vocabulary size reasonable (typically 50K-100K tokens)
- Consider your domain when building vocabulary
Phase 2: Data Sampling
Goal: Sample input data and prepare it for training by separating into sequences of specific length and generating expected responses.
Key Concepts
- Sequence length: Fixed number of tokens per training example
- Context window: How much history the model sees
- Target generation: What the model should predict (next token)
Implementation Steps
- Load and concatenate all training text
- Tokenize the entire corpus
- Split into sequences of fixed length (e.g., 1024 tokens)
- Create input/target pairs:
- Input: tokens [0, 1, 2, ..., n-1]
- Target: tokens [1, 2, 3, ..., n]
- Batch sequences for efficient training
Best Practices
- Use sequence lengths that fit your GPU memory
- Shuffle sequences between epochs
- Consider overlapping sequences for more training data
- Balance dataset across domains if using mixed data
Phase 3: Token Embeddings
Goal: Assign each token a vector representation of desired dimensions. Each word becomes a point in X-dimensional space.
Key Concepts
- Embedding dimension: Size of the vector (e.g., 512, 1024, 4096)
- Learnable parameters: Embeddings are initialized randomly and trained
- Position embeddings: Additional vectors encoding word position
Implementation Steps
- Initialize token embeddings randomly (vocab_size × embedding_dim)
- Initialize position embeddings randomly (max_seq_len × embedding_dim)
- Combine embeddings: token_embedding + position_embedding
- Train embeddings alongside model parameters
Position Embedding Types
- Absolute: Fixed position encoding (simple, effective)
- Relative: Encodes distance between tokens
- Rotary: Rotates embeddings based on position (RoPE)
Best Practices
- Embedding dimension should match model hidden size
- Use learned embeddings rather than fixed ones
- Consider sinusoidal position embeddings for extrapolation
Phase 4: Attention Mechanisms
Goal: Apply attention layers to capture relationships between words in the sentence.
Key Concepts
- Self-attention: Each token attends to all tokens in the sequence
- Query, Key, Value: Three projections for attention computation
- Multi-head attention: Multiple attention heads in parallel
- Causal masking: Prevents attending to future tokens (for training)
Implementation Steps
- Project embeddings to Q, K, V matrices
- Compute attention scores: Q × K^T / sqrt(d_k)
- Apply causal mask (for decoder-only models)
- Softmax to get attention weights
- Weighted sum: attention_weights × V
- Combine heads and project back
Attention Formula
Attention(Q, K, V) = softmax(QK^T / sqrt(d_k))V
Best Practices
- Use multi-head attention (8-16 heads typical)
- Apply layer normalization before and after attention
- Use residual connections around attention blocks
- Consider flash attention for efficiency
Phase 5: LLM Architecture
Goal: Develop the full LLM architecture by combining all components.
Standard Transformer Decoder Architecture
Input → Token Embedding → Position Embedding → [N × (Attention → MLP)] → Output Projection → Logits
Components
- Embedding Layer: Token + Position embeddings
- N Transformer Blocks:
- Multi-head self-attention
- Layer normalization
- Feed-forward MLP (2-4x hidden size)
- Layer normalization
- Output Projection: Hidden size → vocabulary size
- Loss Function: Cross-entropy on next token prediction
Implementation Steps
- Define model class with all layers
- Implement forward pass through all components
- Implement training loop with loss computation
- Implement generation (sampling, beam search, etc.)
- Add saving/loading for model checkpoints
Best Practices
- Use pre-norm architecture (norm before attention/MLP)
- Initialize weights carefully (e.g., Xavier, He initialization)
- Use gradient clipping to prevent exploding gradients
- Implement mixed precision training for efficiency
Phase 6: Pre-training
Goal: Train the model from scratch using the defined architecture, loss functions, and optimizer.
Training Loop
for epoch in epochs:
for batch in dataloader:
# Forward pass
logits = model(input_tokens)
# Compute loss
loss = cross_entropy(logits, target_tokens)
# Backward pass
loss.backward()
# Update weights
optimizer.step()
optimizer.zero_grad()
Key Hyperparameters
- Learning rate: 1e-4 to 3e-4 (with warmup)
- Batch size: Depends on GPU memory (effective batch size 1024-4096)
- Optimizer: AdamW with weight decay (0.01-0.1)
- Learning rate schedule: Cosine decay or linear warmup + decay
- Gradient accumulation: For larger effective batch sizes
Best Practices
- Use learning rate warmup (first 10% of steps)
- Monitor training loss and perplexity
- Save checkpoints regularly
- Use gradient checkpointing for memory efficiency
- Consider distributed training for large models
Phase 7: Fine-tuning
7.0 LoRA (Low-Rank Adaptation)
Goal: Reduce computation needed for fine-tuning by training only small adapter matrices.
How LoRA Works
- Freeze pre-trained weights
- Add small rank-r matrices to attention layers
- Train only the LoRA parameters
- Merge LoRA weights with base model for inference
Implementation Steps
- Freeze base model parameters
- Add LoRA adapters to attention Q, V, (optionally K, O)
- Train only LoRA parameters
- Merge weights for deployment
Best Practices
- Rank r: 8-64 (higher for more capacity)
- Alpha: Scaling factor (typically 2× rank)
- Apply to attention layers primarily
- Use lower learning rate than pre-training
7.1 Fine-tuning for Classification
Goal: Adapt pre-trained model to classify text into categories.
Implementation Steps
- Load pre-trained model (frozen or partially unfrozen)
- Add classification head on top of embeddings
- Prepare labeled dataset with categories
- Train with cross-entropy loss on labels
- Evaluate with accuracy, F1, etc.
Best Practices
- Use mean pooling or [CLS] token for classification
- Fine-tune last 1-2 layers initially
- Use smaller learning rate than pre-training
- Consider few-shot learning for limited data
7.2 Fine-tuning for Instruction Following
Goal: Adapt pre-trained model to follow instructions (chat, tasks, etc.).
Implementation Steps
- Prepare instruction dataset (instruction, input, output format)
- Format examples with special tokens:
<instruction> {instruction} <input> {input} <output> {output}
- Train on formatted data with next-token prediction
- Evaluate on instruction following benchmarks
Best Practices
- Use diverse instruction templates
- Include both simple and complex instructions
- Consider supervised fine-tuning (SFT) before RLHF
- Use quality datasets (e.g., Alpaca, Dolly)
- Monitor for instruction following vs. memorization
Common Issues and Solutions
| Issue |
Solution |
| Training loss not decreasing |
Check learning rate, batch size, data quality |
| Model generates repetitive text |
Adjust temperature, use top-k/top-p sampling |
| Out of memory |
Reduce batch size, use gradient checkpointing |
| Slow training |
Use mixed precision, flash attention |
| Poor generalization |
More data, regularization, better architecture |
Next Steps
After completing these phases, you can:
- Deploy your model for inference
- Optimize with quantization, pruning
- Scale to larger datasets and models
- Experiment with different architectures
- Fine-tune for your specific use case
References
- Manning Book: "Build a Large Language Model from Scratch"
- Original Transformer Paper: "Attention Is All You Need"
- LoRA Paper: "LoRA: Low-Rank Adaptation of Large Language Models"
- Various implementation guides and tutorials
1---2name: llm-training-guide3description: Guide for building and training large language models from scratch. Use this skill whenever the user wants to understand LLM training concepts, implement tokenization, data sampling, embeddings, attention mechanisms, model architecture, pre-training, or fine-tuning workflows. Trigger on mentions of LLM training, building models from scratch, tokenization, embeddings, attention, pre-training, fine-tuning, LoRA, or any LLM development task.4metadata:5 source: https://skillmd.com/skills/abelrguezr/llm-training-guide6---78# LLM Training Guide910A comprehensive guide for building and training large language models from scratch, based on the Manning book "Build a Large Language Model from Scratch".1112## Overview1314This skill covers the complete LLM training pipeline:15161. **Tokenization** - Converting text to token IDs172. **Data Sampling** - Preparing training data183. **Token Embeddings** - Vector representations194. **Attention Mechanisms** - Capturing word relationships205. **LLM Architecture** - Full model structure216. **Pre-training** - Training from scratch227. **Fine-tuning** - Adapting for specific tasks2324## Phase 1: Tokenization2526**Goal**: Divide input text into tokens (IDs) in a meaningful way.2728### Key Concepts2930- **Tokens**: The basic units the model processes (can be characters, words, or subwords)31- **Vocabulary**: The set of all unique tokens32- **Token IDs**: Numeric identifiers for each token in the vocabulary3334### Implementation Steps35361. **Build vocabulary** from your training corpus372. **Create token-to-ID mapping** (tokenizer)383. **Create ID-to-token mapping** (for decoding)394. **Encode text** → convert to token IDs405. **Decode IDs** → convert back to text4142### Best Practices4344- Use subword tokenization (like BPE or WordPiece) for better coverage45- Include special tokens: `<pad>`, `<unk>`, `<bos>`, `<eos>`46- Keep vocabulary size reasonable (typically 50K-100K tokens)47- Consider your domain when building vocabulary4849## Phase 2: Data Sampling5051**Goal**: Sample input data and prepare it for training by separating into sequences of specific length and generating expected responses.5253### Key Concepts5455- **Sequence length**: Fixed number of tokens per training example56- **Context window**: How much history the model sees57- **Target generation**: What the model should predict (next token)5859### Implementation Steps60611. **Load and concatenate** all training text622. **Tokenize** the entire corpus633. **Split into sequences** of fixed length (e.g., 1024 tokens)644. **Create input/target pairs**:65 - Input: tokens [0, 1, 2, ..., n-1]66 - Target: tokens [1, 2, 3, ..., n]675. **Batch sequences** for efficient training6869### Best Practices7071- Use sequence lengths that fit your GPU memory72- Shuffle sequences between epochs73- Consider overlapping sequences for more training data74- Balance dataset across domains if using mixed data7576## Phase 3: Token Embeddings7778**Goal**: Assign each token a vector representation of desired dimensions. Each word becomes a point in X-dimensional space.7980### Key Concepts8182- **Embedding dimension**: Size of the vector (e.g., 512, 1024, 4096)83- **Learnable parameters**: Embeddings are initialized randomly and trained84- **Position embeddings**: Additional vectors encoding word position8586### Implementation Steps87881. **Initialize token embeddings** randomly (vocab_size × embedding_dim)892. **Initialize position embeddings** randomly (max_seq_len × embedding_dim)903. **Combine embeddings**: token_embedding + position_embedding914. **Train embeddings** alongside model parameters9293### Position Embedding Types9495- **Absolute**: Fixed position encoding (simple, effective)96- **Relative**: Encodes distance between tokens97- **Rotary**: Rotates embeddings based on position (RoPE)9899### Best Practices100101- Embedding dimension should match model hidden size102- Use learned embeddings rather than fixed ones103- Consider sinusoidal position embeddings for extrapolation104105## Phase 4: Attention Mechanisms106107**Goal**: Apply attention layers to capture relationships between words in the sentence.108109### Key Concepts110111- **Self-attention**: Each token attends to all tokens in the sequence112- **Query, Key, Value**: Three projections for attention computation113- **Multi-head attention**: Multiple attention heads in parallel114- **Causal masking**: Prevents attending to future tokens (for training)115116### Implementation Steps1171181. **Project embeddings** to Q, K, V matrices1192. **Compute attention scores**: Q × K^T / sqrt(d_k)1203. **Apply causal mask** (for decoder-only models)1214. **Softmax** to get attention weights1225. **Weighted sum**: attention_weights × V1236. **Combine heads** and project back124125### Attention Formula126127```128Attention(Q, K, V) = softmax(QK^T / sqrt(d_k))V129```130131### Best Practices132133- Use multi-head attention (8-16 heads typical)134- Apply layer normalization before and after attention135- Use residual connections around attention blocks136- Consider flash attention for efficiency137138## Phase 5: LLM Architecture139140**Goal**: Develop the full LLM architecture by combining all components.141142### Standard Transformer Decoder Architecture143144```145Input → Token Embedding → Position Embedding → [N × (Attention → MLP)] → Output Projection → Logits146```147148### Components1491501. **Embedding Layer**: Token + Position embeddings1512. **N Transformer Blocks**:152 - Multi-head self-attention153 - Layer normalization154 - Feed-forward MLP (2-4x hidden size)155 - Layer normalization1563. **Output Projection**: Hidden size → vocabulary size1574. **Loss Function**: Cross-entropy on next token prediction158159### Implementation Steps1601611. **Define model class** with all layers1622. **Implement forward pass** through all components1633. **Implement training loop** with loss computation1644. **Implement generation** (sampling, beam search, etc.)1655. **Add saving/loading** for model checkpoints166167### Best Practices168169- Use pre-norm architecture (norm before attention/MLP)170- Initialize weights carefully (e.g., Xavier, He initialization)171- Use gradient clipping to prevent exploding gradients172- Implement mixed precision training for efficiency173174## Phase 6: Pre-training175176**Goal**: Train the model from scratch using the defined architecture, loss functions, and optimizer.177178### Training Loop179180```python181for epoch in epochs:182 for batch in dataloader:183 # Forward pass184 logits = model(input_tokens)185 186 # Compute loss187 loss = cross_entropy(logits, target_tokens)188 189 # Backward pass190 loss.backward()191 192 # Update weights193 optimizer.step()194 optimizer.zero_grad()195```196197### Key Hyperparameters198199- **Learning rate**: 1e-4 to 3e-4 (with warmup)200- **Batch size**: Depends on GPU memory (effective batch size 1024-4096)201- **Optimizer**: AdamW with weight decay (0.01-0.1)202- **Learning rate schedule**: Cosine decay or linear warmup + decay203- **Gradient accumulation**: For larger effective batch sizes204205### Best Practices206207- Use learning rate warmup (first 10% of steps)208- Monitor training loss and perplexity209- Save checkpoints regularly210- Use gradient checkpointing for memory efficiency211- Consider distributed training for large models212213## Phase 7: Fine-tuning214215### 7.0 LoRA (Low-Rank Adaptation)216217**Goal**: Reduce computation needed for fine-tuning by training only small adapter matrices.218219### How LoRA Works220221- Freeze pre-trained weights222- Add small rank-r matrices to attention layers223- Train only the LoRA parameters224- Merge LoRA weights with base model for inference225226### Implementation Steps2272281. **Freeze base model** parameters2292. **Add LoRA adapters** to attention Q, V, (optionally K, O)2303. **Train only LoRA parameters**2314. **Merge weights** for deployment232233### Best Practices234235- Rank r: 8-64 (higher for more capacity)236- Alpha: Scaling factor (typically 2× rank)237- Apply to attention layers primarily238- Use lower learning rate than pre-training239240### 7.1 Fine-tuning for Classification241242**Goal**: Adapt pre-trained model to classify text into categories.243244### Implementation Steps2452461. **Load pre-trained model** (frozen or partially unfrozen)2472. **Add classification head** on top of embeddings2483. **Prepare labeled dataset** with categories2494. **Train with cross-entropy loss** on labels2505. **Evaluate** with accuracy, F1, etc.251252### Best Practices253254- Use mean pooling or [CLS] token for classification255- Fine-tune last 1-2 layers initially256- Use smaller learning rate than pre-training257- Consider few-shot learning for limited data258259### 7.2 Fine-tuning for Instruction Following260261**Goal**: Adapt pre-trained model to follow instructions (chat, tasks, etc.).262263### Implementation Steps2642651. **Prepare instruction dataset** (instruction, input, output format)2662. **Format examples** with special tokens:267 ```268 <instruction> {instruction} <input> {input} <output> {output}269 ```2703. **Train on formatted data** with next-token prediction2714. **Evaluate** on instruction following benchmarks272273### Best Practices274275- Use diverse instruction templates276- Include both simple and complex instructions277- Consider supervised fine-tuning (SFT) before RLHF278- Use quality datasets (e.g., Alpaca, Dolly)279- Monitor for instruction following vs. memorization280281## Common Issues and Solutions282283| Issue | Solution |284|-------|----------|285| Training loss not decreasing | Check learning rate, batch size, data quality |286| Model generates repetitive text | Adjust temperature, use top-k/top-p sampling |287| Out of memory | Reduce batch size, use gradient checkpointing |288| Slow training | Use mixed precision, flash attention |289| Poor generalization | More data, regularization, better architecture |290291## Next Steps292293After completing these phases, you can:2942951. **Deploy** your model for inference2962. **Optimize** with quantization, pruning2973. **Scale** to larger datasets and models2984. **Experiment** with different architectures2995. **Fine-tune** for your specific use case300301## References302303- Manning Book: "Build a Large Language Model from Scratch"304- Original Transformer Paper: "Attention Is All You Need"305- LoRA Paper: "LoRA: Low-Rank Adaptation of Large Language Models"306- Various implementation guides and tutorials