Karpathy Task Routing
Karpathy: "Software 2.0 easily automates what you can verify" (karpathy.bearblog.dev/verifiability, 17 Nov 2025). Set autonomy per task from two questions: can I verify the output cheaply, and is this on-distribution or novel?
The human is the bottleneck of the generation-verification loop, so task size scales to verification bandwidth, not generation bandwidth.
Decision procedure
- Verifiability test. Is there an objective, machine-checkable success signal — test, metric, diff, command output? If not, restructure the task until there is one, or do not delegate: judgment and taste stay human. Grant full autonomy only where the check is automatic — give objective, metric, and boundaries, then leave the inner loop.
- Distribution test. Boilerplate and internet-common patterns: delegate wholesale. Novel, dense, or unconventional code: hand-write with autocomplete, or document every deviation from convention in context before delegating. On nanochat, Karpathy reported agents were "net unhelpful" and gave the reason that the repo was possibly too far off the data distribution (nanochat HN thread, Oct 2025). Separately, on ordinary repos he reports bloated abstractions, dead code left behind, and edits to code orthogonal to the task (X status/2015883857489522876, 26 Jan 2026, agent-coding notes).
- Set the slider, do not max it. Pick a rung from the slider below and name it before starting.
- Granularity. Delegate at macro level — implement this feature, refactor this subsystem, research this library — declaratively: goals plus success criteria, not step-by-step imperatives. Specific beats vague, because a failed verification costs more than a longer prompt. Co-write a spec first for anything substantial; the spec is the durable artifact. Karpathy's version: with LLM agents there is less need to share the specific code or app, you share the idea (X status/2040470801506541998, Apr 2026, idea-file thread).
- Parallelism. For parameter or config search, one sequential agent doing binary search beats parallel sweeps, which waste compute; for open-ended work, run a planner ("chief scientist") agent that emits concrete tasks for worker agents (HN comment 47293311, autoresearch thread, 8 Mar 2026). Run parallel agents only on disjoint functionality with no shared files or modules
(house rule).
- Escalation ladder when stuck. Climb one rung at a time; do not jump straight to the top.
- Autocomplete, then a coding agent on the same task with a sharper success criterion.
- Strongest reasoning model, with the FULL problem context pasted in — the failing code, the exact error, the docs.
- For high-stakes questions, a council of frontier models; their cross-ranking is input, your own qualitative read is the final ranking.
- On failure, assume skill issue first. Karpathy's framing: when agent workflows do not work, it all feels like a skill issue (No Priors ep. 80, 20 Mar 2026). Iterate the prompt, the context, the decomposition, and the harness before concluding the model cannot do it. To turn an observed failure into a standing rule, use
karpathy-context-engineering — that skill owns the codify-mistakes rule.
Autonomy slider
- Autocomplete — routine code you already know how to write.
- Scoped agent task — a well-specified chunk with one named success criterion.
- Agent loop — an objective metric exists; the agent iterates unsupervised against it.
- Vibe mode, Accept-All, diffs unread — throwaway prototypes ONLY, consciously entered AND consciously exited. Karpathy: "You are not allowed to introduce vulnerabilities because of vibe coding." (karpathy.bearblog.dev/sequoia-ascent-2026, 30 Apr 2026; Karpathy states in that post that he had a model produce the cleaned transcript, so its wording is LLM-cleaned rather than raw speech).
Task -> setting
| Task |
Setting |
| CRUD endpoint, tests, glue code |
Agent, wholesale |
| Novel algorithm, precise architecture |
Hand-write + autocomplete |
| Unfamiliar language or framework |
Agent (best leverage-to-risk) |
| Kernel or perf tuning with a metric |
Autonomous loop |
| Naming, API design, strategy |
Human |
| Weekend prototype |
Vibe mode, entered and exited on purpose |
Model & tool routing
| Need |
Route |
| Routine, low-stakes query |
Cheap fast model |
| Math, debugging, multi-step logic |
Reasoning model; accept the latency |
| High-stakes hard question |
Council of frontier models (his council ships with four: gpt-5.1, gemini-3-pro-preview, claude-sonnet-4.5, grok-4); your own read is final |
| Any arithmetic or data work |
Force code execution; never in-head model math |
| Post-cutoff or dynamic facts |
Force web search and click the citations |
| Multi-source survey |
Deep research as a first draft; read the primary sources it cites |
| Reading a paper |
Load the current chapter and interrogate it section by section |
| Repeated preferences |
Memory or custom instructions, pruned periodically |
These rows follow the practice described in Karpathy's "How I use LLMs" (youtube EWvNQjAaOHw, 28 Feb 2025). The transcript could not be retrieved for verification, so treat every row as sourced paraphrase, not quotation. The four council models are from X status/1992381094667411768 (22 Nov 2025, llm-council announcement); in that thread the council ranked GPT-5.1 top while he preferred Gemini 3's answer.
Excuse -> reality
| Excuse |
Reality |
| Parallel will be faster |
Only if the agents touch disjoint files; otherwise they interfere and you pay twice |
| The model just can't do this |
Skill issue first: prompt, context, decomposition, harness |
| I'll review the output after |
Can you actually check it? If not, it was never delegable |
| One clean run, ship it |
works.any() is not works.all(); budget for the march of nines |
| A 10k-line diff is fine, it's generated |
You cannot audit it, so nobody has |
Everyday LLM usage and learning
When the question is how to work with AI day to day, how to structure a prompt, when to build a throwaway single-use tool, or how to learn a subject — including a bare "help me learn X", where the answer is a study approach and not a summary — read references/everyday-llm-usage.md before answering.
It also holds the working habits and learning practices absorbed from the retired karpathy-llm-usage, plus the three routing rows too rare for the table above: dense reading turned into audio, voice input, and camera capture.
Sources
Karpathy: karpathy.bearblog.dev/verifiability (17 Nov 2025) for the verifiability test; karpathy.bearblog.dev/sequoia-ascent-2026 (30 Apr 2026, LLM-cleaned transcript) for the vibe-coding quality bar and macro-level delegation; nanochat HN thread (Oct 2025) for "net unhelpful"; X status/2015883857489522876 (26 Jan 2026, agent-coding notes) for the bloat and out-of-scope-edit failure modes; HN comment 47293311 (autoresearch thread, 8 Mar 2026) for bisection-over-sweeps and the chief-scientist planner; No Priors ep. 80 (20 Mar 2026) for skill issue; X status/2040470801506541998 (Apr 2026, idea-file thread) for share-the-idea; X status/1992381094667411768 (22 Nov 2025, llm-council announcement) for the council; "How I use LLMs" (youtube EWvNQjAaOHw, 28 Feb 2025) for the routing table.
Third-party: latent.space/p/s3 (17 Jun 2025) records "Demo is works.any(), product is works.all()" as slide text from the YC Software 3.0 talk, and the march of nines is from the Dwarkesh interview (17 Oct 2025) — both are stated in full in karpathy-verification.
MIT notice for this skill set, covering text adapted from multica-ai/andrej-karpathy-skills (formerly forrestchang/andrej-karpathy-skills): see karpathy-coding-loop/references/examples.md.
1---2name: karpathy-task-routing3description: Use when deciding how to run a task rather than doing it: how much autonomy to give an agent, whether to hand-write instead, whether to parallelize agents, which model or tool tier fits, or what to do when a delegated task keeps failing. Also when the user asks how to use LLMs efficiently, how Karpathy works with AI, or how to learn or study a topic — including bare 'help me learn X' requests.4---56# Karpathy Task Routing78Karpathy: "Software 2.0 easily automates what you can verify" (karpathy.bearblog.dev/verifiability, 17 Nov 2025). Set autonomy per task from two questions: can I verify the output cheaply, and is this on-distribution or novel?9The human is the bottleneck of the generation-verification loop, so task size scales to verification bandwidth, not generation bandwidth.1011## Decision procedure12131. **Verifiability test.** Is there an objective, machine-checkable success signal — test, metric, diff, command output? If not, restructure the task until there is one, or do not delegate: judgment and taste stay human. Grant full autonomy only where the check is automatic — give objective, metric, and boundaries, then leave the inner loop.142. **Distribution test.** Boilerplate and internet-common patterns: delegate wholesale. Novel, dense, or unconventional code: hand-write with autocomplete, or document every deviation from convention in context before delegating. On nanochat, Karpathy reported agents were "net unhelpful" and gave the reason that the repo was possibly too far off the data distribution (nanochat HN thread, Oct 2025). Separately, on ordinary repos he reports bloated abstractions, dead code left behind, and edits to code orthogonal to the task (X status/2015883857489522876, 26 Jan 2026, agent-coding notes).153. **Set the slider, do not max it.** Pick a rung from the slider below and name it before starting.164. **Granularity.** Delegate at macro level — implement this feature, refactor this subsystem, research this library — declaratively: goals plus success criteria, not step-by-step imperatives. Specific beats vague, because a failed verification costs more than a longer prompt. Co-write a spec first for anything substantial; the spec is the durable artifact. Karpathy's version: with LLM agents there is less need to share the specific code or app, you share the idea (X status/2040470801506541998, Apr 2026, idea-file thread).175. **Parallelism.** For parameter or config search, one sequential agent doing binary search beats parallel sweeps, which waste compute; for open-ended work, run a planner ("chief scientist") agent that emits concrete tasks for worker agents (HN comment 47293311, autoresearch thread, 8 Mar 2026). Run parallel agents only on disjoint functionality with no shared files or modules `(house rule)`.186. **Escalation ladder when stuck.** Climb one rung at a time; do not jump straight to the top.19 - Autocomplete, then a coding agent on the same task with a sharper success criterion.20 - Strongest reasoning model, with the FULL problem context pasted in — the failing code, the exact error, the docs.21 - For high-stakes questions, a council of frontier models; their cross-ranking is input, your own qualitative read is the final ranking.227. **On failure, assume skill issue first.** Karpathy's framing: when agent workflows do not work, it all feels like a skill issue (No Priors ep. 80, 20 Mar 2026). Iterate the prompt, the context, the decomposition, and the harness before concluding the model cannot do it. To turn an observed failure into a standing rule, use `karpathy-context-engineering` — that skill owns the codify-mistakes rule.2324## Autonomy slider2526- **Autocomplete** — routine code you already know how to write.27- **Scoped agent task** — a well-specified chunk with one named success criterion.28- **Agent loop** — an objective metric exists; the agent iterates unsupervised against it.29- **Vibe mode, Accept-All, diffs unread** — throwaway prototypes ONLY, consciously entered AND consciously exited. Karpathy: "You are not allowed to introduce vulnerabilities because of vibe coding." (karpathy.bearblog.dev/sequoia-ascent-2026, 30 Apr 2026; Karpathy states in that post that he had a model produce the cleaned transcript, so its wording is LLM-cleaned rather than raw speech).3031## Task -> setting3233| Task | Setting |34|------|---------|35| CRUD endpoint, tests, glue code | Agent, wholesale |36| Novel algorithm, precise architecture | Hand-write + autocomplete |37| Unfamiliar language or framework | Agent (best leverage-to-risk) |38| Kernel or perf tuning with a metric | Autonomous loop |39| Naming, API design, strategy | Human |40| Weekend prototype | Vibe mode, entered and exited on purpose |4142## Model & tool routing4344| Need | Route |45|------|-------|46| Routine, low-stakes query | Cheap fast model |47| Math, debugging, multi-step logic | Reasoning model; accept the latency |48| High-stakes hard question | Council of frontier models (his council ships with four: gpt-5.1, gemini-3-pro-preview, claude-sonnet-4.5, grok-4); your own read is final |49| Any arithmetic or data work | Force code execution; never in-head model math |50| Post-cutoff or dynamic facts | Force web search and click the citations |51| Multi-source survey | Deep research as a first draft; read the primary sources it cites |52| Reading a paper | Load the current chapter and interrogate it section by section |53| Repeated preferences | Memory or custom instructions, pruned periodically |5455These rows follow the practice described in Karpathy's "How I use LLMs" (youtube EWvNQjAaOHw, 28 Feb 2025). The transcript could not be retrieved for verification, so treat every row as sourced paraphrase, not quotation. The four council models are from X status/1992381094667411768 (22 Nov 2025, llm-council announcement); in that thread the council ranked GPT-5.1 top while he preferred Gemini 3's answer.5657## Excuse -> reality5859| Excuse | Reality |60|--------|---------|61| Parallel will be faster | Only if the agents touch disjoint files; otherwise they interfere and you pay twice |62| The model just can't do this | Skill issue first: prompt, context, decomposition, harness |63| I'll review the output after | Can you actually check it? If not, it was never delegable |64| One clean run, ship it | works.any() is not works.all(); budget for the march of nines |65| A 10k-line diff is fine, it's generated | You cannot audit it, so nobody has |6667## Everyday LLM usage and learning6869When the question is how to work with AI day to day, how to structure a prompt, when to build a throwaway single-use tool, or how to learn a subject — including a bare "help me learn X", where the answer is a study approach and not a summary — read `references/everyday-llm-usage.md` before answering.70It also holds the working habits and learning practices absorbed from the retired `karpathy-llm-usage`, plus the three routing rows too rare for the table above: dense reading turned into audio, voice input, and camera capture.7172## Sources7374Karpathy: karpathy.bearblog.dev/verifiability (17 Nov 2025) for the verifiability test; karpathy.bearblog.dev/sequoia-ascent-2026 (30 Apr 2026, LLM-cleaned transcript) for the vibe-coding quality bar and macro-level delegation; nanochat HN thread (Oct 2025) for "net unhelpful"; X status/2015883857489522876 (26 Jan 2026, agent-coding notes) for the bloat and out-of-scope-edit failure modes; HN comment 47293311 (autoresearch thread, 8 Mar 2026) for bisection-over-sweeps and the chief-scientist planner; No Priors ep. 80 (20 Mar 2026) for skill issue; X status/2040470801506541998 (Apr 2026, idea-file thread) for share-the-idea; X status/1992381094667411768 (22 Nov 2025, llm-council announcement) for the council; "How I use LLMs" (youtube EWvNQjAaOHw, 28 Feb 2025) for the routing table.75Third-party: latent.space/p/s3 (17 Jun 2025) records "Demo is works.any(), product is works.all()" as slide text from the YC Software 3.0 talk, and the march of nines is from the Dwarkesh interview (17 Oct 2025) — both are stated in full in `karpathy-verification`.76MIT notice for this skill set, covering text adapted from multica-ai/andrej-karpathy-skills (formerly forrestchang/andrej-karpathy-skills): see `karpathy-coding-loop/references/examples.md`.
yshms