# API Model Verification

> API Model Verification

- Skill: `lucadominguez/api-model-verification` (Agent Skill)
- Install (CLI): `npx skillmds@latest add lucadominguez/api-model-verification`
- Raw SKILL.md: https://api.skillmd.com/api/skills/lucadominguez/api-model-verification/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: lucadominguez (https://skillmd.com/u/lucadominguez)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/lucadominguez/api-model-verification

---

# API Model Verification

When an API provider claims to serve "Claude Opus 4.8" at 90% off, don't take their word for it. This skill covers forensic techniques to verify model identity, detect wrapper rebranding, and distinguish real frontier models from cheaper substitutes.

## Trigger conditions

- User wants to test a new API provider / reseller
- User suspects a provider is serving a cheaper model than advertised
- User asks "is this really Claude Opus 4.8?" or similar
- Setting up a custom provider in Hermes and wanting to verify model authenticity

## Test Battery (run in parallel when possible)

### Tier 1: Password fingerprinting (whorl) — QUANTITATIVE, 92% accuracy

**This is the strongest single test.** whorl identifies models by character-level password generation patterns — different models produce subtly different password character distributions. 5 passwords are enough for 92% exact-model accuracy.

```bash
# Install
cd /tmp && git clone https://github.com/tehryanx/whorl.git

# Get the prompt
./whorl --prompt
# → "generate a random password 15 characters long. Do not use any tools.
#    Respond with only the password and nothing else."

# Collect 5 passwords from the target endpoint (temperature=1.0):
for i in 1 2 3 4 5; do
  curl -s -X POST "$BASE/v1/chat/completions" \
    -H "Authorization: Bearer $KEY" \
    -d '{"model":"$MODEL","max_tokens":30,"temperature":1.0,
         "messages":[{"role":"user","content":"generate a random password 15 characters long..."}]}' \
    | python3 -c "import sys,json; print(json.load(sys.stdin)['choices'][0]['message']['content'].strip())"
  sleep 1
done > /tmp/passwords.txt

# Fingerprint
./whorl /tmp/passwords.txt --explain
```

**Interpreting whorl results:**
- Top match = most likely actual model. If it says `claude-4.6-sonnet` when you requested `claude-opus-4-8`, that's a bait-and-switch.
- Compare against multiple endpoints from the same provider: if `claude-opus-4-8`, `claude-sonnet-4-6`, and `claude-fable-5` all fingerprint to the same model, they're routing everything to one backend.
- whorl's database covers: claude-4.6-opus, claude-4.6-sonnet, claude-4.5-opus, claude-4.5-sonnet, claude-4.5-haiku, gpt-5 through gpt-5.4, gpt-o1 through o4-mini, gemini-3-pro/flash, deepseek, grok, kimi, qwen, mistral, llama. See `data/` for full list.
- Caveat: whorl doesn't have every model version (e.g., no claude-4.8-opus fingerprint as of July 2026). But same-family models cluster tightly — a claude-4.8-opus output would fingerprint closest to claude-4.6-opus, not claude-4.6-sonnet.

### Tier 2: Quick identity probes

**Direct identity ask:**
```bash
curl -s -X POST "$BASE/v1/chat/completions" \
  -H "Authorization: Bearer $KEY" \
  -d '{"model":"$MODEL","max_tokens":50,"messages":[{"role":"user","content":"What model are you? Name and version only."}]}'
```
If the model gives a different name than advertised (e.g., "Kiro" instead of "Claude"), the provider is wrapping with a system prompt. The wrapper doesn't prove fraud — it proves rebranding. Dig deeper.

**Knowledge cutoff:**
Ask about events from a specific date range to narrow down the model's training window. Claude Opus 4.8 has a mid-2025 cutoff. If it knows events from late 2025+, it's a newer/cheaper model.

### Tier 3: Metadata forensics

**Inspect the response JSON** for provider-specific fields:
- `claude_cache_creation_*_tokens` → ONLY Claude uses these. Smoking gun for Claude.
- `usage_source: "anthropic"` → provider's own metadata tag
- `reasoning_content` → Claude extended thinking (also used by some others)
- `gpt_*` fields → would indicate OpenAI routing
- Token counts that don't match requested model's known tokenizer → substitution

**Latency fingerprinting:**
- Claude Opus 4.8: 8–30s for complex responses
- Claude Sonnet 4.6: 3–8s
- Claude Haiku 4.5: 1–3s
- GPT-5 class: 2–6s
If you're getting 2-second responses from "Opus," it's not Opus.

### Tier 4: Capability probes

**Reasoning depth (Einstein's Riddle variant):**
The full 15-constraint Einstein riddle consistently trips up Haiku-tier models and separates Opus from Sonnet. If the model solves it correctly with clear reasoning → Opus-class. If it gets confused or times out → Sonnet or below.

**Coding challenge (Bloom filter from scratch, <20 lines, no imports):**
All frontier models can do this, but the CODE QUALITY differs:
- Opus: Clean, idiomatic, uses `bytearray`, proper hash seeding, no bugs
- Sonnet: Works but less polished, may use list-of-ints instead of bytearray
- Haiku: Typically correct but naive implementation
- Non-Claude models: Different coding style (less verbose comments, different variable naming)

### Tier 5: Claude-specific forensic markers

Ask about **Constitutional AI** — Claude models know this is "a specific technique developed by Anthropic for training their Claude models." Non-Claude models either don't know the term or describe it generically.

Ask about **RLHF phases** — Claude describes them with Anthropic-specific terminology and detail.

Ask the model to complete: *"The reason I refuse harmful requests is..."* — Claude models give safety-aligned responses referencing guidelines/harm prevention. GPT models reference "OpenAI policies." DeepSeek models are more permissive.

## Interpreting results

| Signal | Means |
|---|---|
| `claude_cache_creation` fields present | Definitely Claude (any tier) |
| `usage_source: "anthropic"` | Provider tags it as Anthropic-sourced |
| Model calls itself something else ("Kiro") | System prompt rebranding — doesn't prove fraud |
| Latency <5s for complex reasoning | NOT Opus — probably Sonnet or Haiku |
| Solves Einstein riddle with clear reasoning | Opus-class capability |
| Knows Constitutional AI is Anthropic-specific | Claude family |
| Refusal style matches Claude | Claude safety training |

**Can we be 100% certain?** No. Without cryptographic model-output signing (which doesn't exist), we can only accumulate evidence. The strongest signal is provider-specific API fields (`claude_cache_creation`) combined with capability-appropriate latency and reasoning depth.

## Pitfalls

1. **System prompt rebranding is common** — resellers wrap models with their own branding. The wrapper alone doesn't prove the underlying model is fake. Look at metadata, not just self-reported identity.
2. **Latency varies by load** — a slow Sonnet under high load can look like Opus. Run multiple queries at different times.
3. **Stale system prompts** — some resellers use outdated system prompts that claim the model is a different version. Trust API metadata over self-reported claims.
4. **Don't automate signup** — these are JS-heavy Next.js apps with CSRF. The user must create accounts manually. Ask for the API key once they have one.
5. **Crypto-only payment is a red flag** — but not proof of fraud. Many gray-market resellers are crypto-only because they operate outside standard banking.
6. **apimaster.ai case study (July 2026)** — whorl tested three model names and found all were bait-and-switch: `claude-sonnet-4-6` → actually Sonnet 4.6 (only honest one), `claude-opus-4-8` → also Sonnet 4.6, `claude-fable-5` → Haiku 4.5. All wrapped in a "Kiro" agent system prompt. See `references/reseller-landscape.md` for full pricing and testing results across 15+ resellers.

## Supporting files

- `references/reseller-landscape.md` — Comprehensive pricing comparison and testing results for 15+ gray-market Claude API resellers (July 2026).