Jiachen-T-Wang Alpaca: A Strong, Replicable Instruction-Following Model
One-line decision
Use this skill when you want to distill instruction-following capability from a strong LLM to a weaker one using 52K generated instruction samples. Avoid it when you can directly use the strong LLM or have human-written instructions.
Skill metadata
- Skill type: instruction-data-distillation
- Paper kind: operational-method
- Actionability: high
- Evidence quality: full_paper
Goal
Distill instruction-following capability from GPT-3.5 to LLaMA-7B by generating 52K instruction-following samples using the Self-Instruct approach.
Problem signature
- Modality: text instruction-response pairs generated by GPT-3.5.
- Data state: 52K instruction samples generated by GPT-3.5 from 175 seeds.
- Scale regime: 52K instruction-response pairs.
- Model requirement: GPT-3.5 for generation; LLaMA-7B for fine-tuning.
Use when
- You want to create instruction data cheaply using a strong LLM.
- You need to fine-tune a smaller model for instruction following.
- You want a cost-effective alternative to human annotation.
Do not use when
- You can use the strong LLM directly.
- You have abundant human-annotated instruction data.
- You need domain expertise the LLM lacks.
Required inputs
- strong_llm: GPT-3.5 or equivalent for generating instruction data.
- seed_instructions: 175 seed instruction tasks.
- base_model: LLaMA-7B or equivalent for fine-tuning.
Optional inputs
- quality_filter: Filter for removing low-quality generated instructions.
Outputs
- alpaca_data: 52K instruction-response pairs from GPT-3.5.
- alpaca_model: LLaMA-7B fine-tuned on the generated data.
Assumptions and prerequisites
- Strong LLM outputs provide effective supervision for smaller models.
- 52K samples are sufficient for basic instruction following.
- The Self-Instruct approach scales to production quality.
Procedure
- Generate instructions with GPT-3.5 Action: Use GPT-3.5 to generate 52K instruction-response pairs following Self-Instruct. Why: GPT-3.5 produces reasonable quality instruction data. Note: See paper for details.
- Quality filter Action: Remove obviously low-quality or malformed samples. Why: Basic filtering improves data quality. Note: See paper for details.
- Fine-tune LLaMA-7B Action: Fine-tune the base LLaMA model on the 52K samples. Why: Creates an instruction-following model. Note: See paper for details.
- Evaluate instruction following Action: Test on instruction-following benchmarks. Why: Validates the distillation quality. Note: See paper for details.
Parameters to set
- generation_cost — Role: API cost for generating data. How to set: ~$500 for 52K samples with GPT-3.5. Default/range: ~$500. Effect: Very cost-effective compared to human annotation.
- sample_count — Role: Number of instruction samples. How to set: 52K for basic instruction following. Default/range: 52K. Effect: More samples improve instruction diversity.
Validation checks
- The fine-tuned model should follow diverse instructions.
- Quality should approach that of the teacher model on simple tasks.
- The approach should be reproducible at low cost.
Failure modes
- Generated data inherits biases from GPT-3.5.
- 52K samples may not cover complex instruction types.
- The quality gap between teacher and student may be significant for hard tasks.
Adaptation notes for VLM training
- The Alpaca recipe has been widely replicated for VLM instruction data.
- Replace GPT-3.5 with Claude for data generation.
- Scale to more data for improved instruction following.
Implementation notes
- Use the Alpaca data generation code for reproducibility.
- Monitor per-task quality during fine-tuning.
- Compare against the teacher model on diverse tasks.
Evidence from the paper
- Alpaca creates an instruction-following LLaMA-7B for ~$500 in API costs.
- 52K GPT-3.5-generated samples enable reasonable instruction following.
- The approach demonstrates cost-effective LLM distillation.
- Alpaca has inspired numerous open-source instruction-tuning efforts.
Source paper
- Title: Alpaca: A Strong, Replicable Instruction-Following Model
- Year: 2023
- Venue: Stanford
- Paper ID: stanford-alpaca-2023
- URL: https://crfm.stanford.edu/2023/03/13/alpaca.html
- arXiv ID: N/A