# Alpaca A Strong Replicable Instruction Following Model Stanford Alpaca 2023

> Alpaca: A Strong, Replicable Instruction-Following Model

- Skill: `jiachen-t-wang/alpaca-a-strong-replicable-instruction-following-model-stanf` (Agent Skill)
- Install (CLI): `npx skillmds add jiachen-t-wang/alpaca-a-strong-replicable-instruction-following-model-stanf`
- Raw SKILL.md: https://api.skillmd.com/api/skills/jiachen-t-wang/alpaca-a-strong-replicable-instruction-following-model-stanf/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: Jiachen-T-Wang (https://skillmd.com/u/jiachen-t-wang)
- Updated: 2026-08-19
- Page: https://skillmd.com/skills/jiachen-t-wang/alpaca-a-strong-replicable-instruction-following-model-stanf

---

# Alpaca: A Strong, Replicable Instruction-Following Model

## One-line decision
Use this skill when you want to distill instruction-following capability from a strong LLM to a weaker one using 52K generated instruction samples. Avoid it when you can directly use the strong LLM or have human-written instructions.

## Skill metadata
- **Skill type**: instruction-data-distillation
- **Paper kind**: operational-method
- **Actionability**: high
- **Evidence quality**: full_paper

## Goal
Distill instruction-following capability from GPT-3.5 to LLaMA-7B by generating 52K instruction-following samples using the Self-Instruct approach.

## Problem signature
- Modality: text instruction-response pairs generated by GPT-3.5.
- Data state: 52K instruction samples generated by GPT-3.5 from 175 seeds.
- Scale regime: 52K instruction-response pairs.
- Model requirement: GPT-3.5 for generation; LLaMA-7B for fine-tuning.

## Use when
- You want to create instruction data cheaply using a strong LLM.
- You need to fine-tune a smaller model for instruction following.
- You want a cost-effective alternative to human annotation.

## Do not use when
- You can use the strong LLM directly.
- You have abundant human-annotated instruction data.
- You need domain expertise the LLM lacks.

## Required inputs
- **strong_llm**: GPT-3.5 or equivalent for generating instruction data.
- **seed_instructions**: 175 seed instruction tasks.
- **base_model**: LLaMA-7B or equivalent for fine-tuning.

## Optional inputs
- **quality_filter**: Filter for removing low-quality generated instructions.

## Outputs
- **alpaca_data**: 52K instruction-response pairs from GPT-3.5.
- **alpaca_model**: LLaMA-7B fine-tuned on the generated data.

## Assumptions and prerequisites
- Strong LLM outputs provide effective supervision for smaller models.
- 52K samples are sufficient for basic instruction following.
- The Self-Instruct approach scales to production quality.

## Procedure
1. **Generate instructions with GPT-3.5**
   Action: Use GPT-3.5 to generate 52K instruction-response pairs following Self-Instruct.
   Why: GPT-3.5 produces reasonable quality instruction data.
   Note: See paper for details.
2. **Quality filter**
   Action: Remove obviously low-quality or malformed samples.
   Why: Basic filtering improves data quality.
   Note: See paper for details.
3. **Fine-tune LLaMA-7B**
   Action: Fine-tune the base LLaMA model on the 52K samples.
   Why: Creates an instruction-following model.
   Note: See paper for details.
4. **Evaluate instruction following**
   Action: Test on instruction-following benchmarks.
   Why: Validates the distillation quality.
   Note: See paper for details.

## Parameters to set
- **generation_cost** — Role: API cost for generating data. How to set: ~$500 for 52K samples with GPT-3.5. Default/range: ~$500. Effect: Very cost-effective compared to human annotation.
- **sample_count** — Role: Number of instruction samples. How to set: 52K for basic instruction following. Default/range: 52K. Effect: More samples improve instruction diversity.

## Validation checks
- The fine-tuned model should follow diverse instructions.
- Quality should approach that of the teacher model on simple tasks.
- The approach should be reproducible at low cost.

## Failure modes
- Generated data inherits biases from GPT-3.5.
- 52K samples may not cover complex instruction types.
- The quality gap between teacher and student may be significant for hard tasks.

## Adaptation notes for VLM training
- The Alpaca recipe has been widely replicated for VLM instruction data.
- Replace GPT-3.5 with Claude for data generation.
- Scale to more data for improved instruction following.

## Implementation notes
- Use the Alpaca data generation code for reproducibility.
- Monitor per-task quality during fine-tuning.
- Compare against the teacher model on diverse tasks.

## Evidence from the paper
- Alpaca creates an instruction-following LLaMA-7B for ~$500 in API costs.
- 52K GPT-3.5-generated samples enable reasonable instruction following.
- The approach demonstrates cost-effective LLM distillation.
- Alpaca has inspired numerous open-source instruction-tuning efforts.

## Source paper
- **Title**: Alpaca: A Strong, Replicable Instruction-Following Model
- **Year**: 2023
- **Venue**: Stanford
- **Paper ID**: stanford-alpaca-2023
- **URL**: https://crfm.stanford.edu/2023/03/13/alpaca.html
- **arXiv ID**: N/A