AI Model Route Evaluator
Choose a route for the user's task, not a universal leaderboard winner. Model
catalogs, aliases, limits, prices, and product entitlements are time-sensitive;
verify them from current primary sources.
Workflow
- Define the job. Resolve the actual task, input/output modalities,
languages, context size, tool/agent needs, latency tolerance, budget, data
sensitivity, local hardware, and whether a free route is a hard constraint.
Ask only for a missing fact that could change the winner.
- Resolve exact identities. Record provider, product/plan or gateway, exact
model ID/version/date, endpoint, region when relevant, and whether a name is
a floating alias. Never treat a gateway alias as a stable model identity.
- Verify capability and access. Use official catalogs, model cards, pricing,
rate-limit/usage documentation, privacy/data-use terms, and product help.
Separate API access from consumer subscription access.
- Select task-fit evidence. Prefer current, version-pinned evaluations that
match the required language, modality, agent/tool use, and difficulty. Do not
infer Turkish or multimodal quality from an English text benchmark alone.
- Measure locally when practical. Run the same representative prompt/input,
record settings, retries, time, tokens/cost, success criteria, and failures.
Do not compare outputs produced under materially different scaffolds as if
only the model changed.
- Assess privacy and operational risk. Never route API keys, private code,
customer data, health/legal/financial records, or personal data through an
untrusted or vaguely governed free endpoint. State retention/training facts
only when verified for that exact product and plan.
- Score by scenario. Give one winner per meaningful task class, a fallback,
and explicit disqualifiers. Use weighted scores only after showing the raw
facts; do not hide a hard requirement inside an average.
- Conclude conditionally. State what to use now, for which task, under which
plan/route, why, when to switch, and what remains uncertain.
Read references/evaluation-scorecard.md
for the comparison schema.
Output contract
## Decision
<task-specific winner, exact route, and one-line reason>
| Route | Exact model/version | Best for | Limits/cost | Privacy | Evidence confidence |
|---|---|---|---|---|---|
## By task
- <task>: <winner> — <reason and disqualifier>
## What to do
<setup or selection steps, fallback, and recheck trigger>
## Uncertainty
<floating aliases, missing Turkish/multimodal evidence, unmeasured latency, or plan ambiguity>
Avoid provider fandom, copied leaderboard rank, guaranteed quality, or a single
winner across unrelated tasks.
1---2name: ai-model-route-evaluator3description: Compare current AI models, providers, gateways, subscriptions, local routes, and free/paid endpoints for a concrete task. Evaluate exact model identity, modality, Turkish and multilingual quality, coding/reasoning, context, tool support, privacy, limits, price, latency, reliability, and benchmark fit; give task-specific winners and uncertainty. Turkish triggers: hangi model daha iyi, ChatGPT/Claude/OpenCode modeli seç, ücretsiz model, API karşılaştır, yerel mi bulut mu.4---56# AI Model Route Evaluator78Choose a route for the user's task, not a universal leaderboard winner. Model9catalogs, aliases, limits, prices, and product entitlements are time-sensitive;10verify them from current primary sources.1112## Workflow13141. **Define the job.** Resolve the actual task, input/output modalities,15 languages, context size, tool/agent needs, latency tolerance, budget, data16 sensitivity, local hardware, and whether a free route is a hard constraint.17 Ask only for a missing fact that could change the winner.182. **Resolve exact identities.** Record provider, product/plan or gateway, exact19 model ID/version/date, endpoint, region when relevant, and whether a name is20 a floating alias. Never treat a gateway alias as a stable model identity.213. **Verify capability and access.** Use official catalogs, model cards, pricing,22 rate-limit/usage documentation, privacy/data-use terms, and product help.23 Separate API access from consumer subscription access.244. **Select task-fit evidence.** Prefer current, version-pinned evaluations that25 match the required language, modality, agent/tool use, and difficulty. Do not26 infer Turkish or multimodal quality from an English text benchmark alone.275. **Measure locally when practical.** Run the same representative prompt/input,28 record settings, retries, time, tokens/cost, success criteria, and failures.29 Do not compare outputs produced under materially different scaffolds as if30 only the model changed.316. **Assess privacy and operational risk.** Never route API keys, private code,32 customer data, health/legal/financial records, or personal data through an33 untrusted or vaguely governed free endpoint. State retention/training facts34 only when verified for that exact product and plan.357. **Score by scenario.** Give one winner per meaningful task class, a fallback,36 and explicit disqualifiers. Use weighted scores only after showing the raw37 facts; do not hide a hard requirement inside an average.388. **Conclude conditionally.** State what to use now, for which task, under which39 plan/route, why, when to switch, and what remains uncertain.4041Read [references/evaluation-scorecard.md](references/evaluation-scorecard.md)42for the comparison schema.4344## Output contract4546```markdown47## Decision48<task-specific winner, exact route, and one-line reason>4950| Route | Exact model/version | Best for | Limits/cost | Privacy | Evidence confidence |51|---|---|---|---|---|---|5253## By task54- <task>: <winner> — <reason and disqualifier>5556## What to do57<setup or selection steps, fallback, and recheck trigger>5859## Uncertainty60<floating aliases, missing Turkish/multimodal evidence, unmeasured latency, or plan ambiguity>61```6263Avoid provider fandom, copied leaderboard rank, guaranteed quality, or a single64winner across unrelated tasks.
yigityildiz0