# Baseline Before Model

> Force a non-model baseline to be built and measured before any LLM is proposed for a task. Use when scoping an AI feature, choosing a model for extraction, lookup, classification or routing, or when someone asks which model to use for a job. Refuses to recommend a model until the boring alternative has a number.

- Skill: `ityaadiii/baseline-before-model` (Agent Skill)
- Install (CLI): `npx skillmds@latest add ityaadiii/baseline-before-model`
- Raw SKILL.md: https://api.skillmd.com/api/skills/ityaadiii/baseline-before-model/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: ityaadiii (https://skillmd.com/u/ityaadiii)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/ityaadiii/baseline-before-model

---


# Run the boring thing first

On an address-resolution task, a deterministic directory lookup with an alias table and
edit distance scored **96.7%**. The language models given the same items and no
reference data scored 65.0% and 53.3%. The lookup ran in 2.8ms at zero cost against
roughly 2,000ms and real spend.

That result is not an argument about model quality. It is an argument about
architecture. A prompt-only model is the wrong tool for a lookup, and no leaderboard
will ever tell you that, because "should this be a model at all" is not a property of
any model on it.

## The refusal

**Do not recommend a model for a task with no measured non-model baseline.** If nobody
has built the boring version, the honest output is "unknown, and here is the baseline to
build first".

## Candidate baselines, in order of how often they win

1. **Lookup against a table or directory** you already own. Extraction and normalisation
   tasks are frequently this in disguise.
2. **Rules or a decision table**, when the logic is genuinely small and stable.
3. **Classical retrieval** (BM25) before any embedding pipeline.
4. **A small local model** before a frontier API model.
5. **The existing process**, measured. Often nobody has ever scored the humans.

## Procedure

1. Write the baseline behind the **same interface** the model would use, so the whole
   evaluation stack runs on it unchanged.
2. Score it on the same items, with the same grader.
3. **State the caveat that travels with the number.** If the baseline holds reference
   data the models were not given, that is lookup versus no-lookup, not lookup versus
   retrieval. Say it plainly. The honest claim is stronger than the sloppy one.
4. Add the missing arm where it matters: the model **with** the reference data in
   context. That is the comparison that decides the architecture.
5. Report cost and latency alongside accuracy. This is usually where the baseline wins
   by two orders of magnitude.

## Output contract

- baseline description and what data it holds
- baseline accuracy, cost, latency
- model accuracy, cost, latency, on identical items
- the caveat, stated in the same breath as the numbers
- whether the model-with-reference-data arm was run, and if not, that it is missing

## What this prevents

Six weeks of prompt engineering on a task a hash map solves better, and a vendor
conversation about which lab to buy from when the answer was to buy from neither.