Maintain Model List

Maintain Model List

The supported model list lives in three places that must stay in sync:

Before Touching Anything

1. Run git status. If the working tree has uncommitted changes that look like unrelated in-progress work, do NOT edit files — report the situation and ask the user for permission first.
2. Run the full live test suite as a baseline: npm run test:live -w @page-agent/llms. API keys come from the repo-root .env (TESTING_OPENROUTER_KEY, TESTING_ALIYUN_KEY, TESTING_DEEPSEEK_KEY); tests skip silently when a key is missing, so check which providers actually ran. Record which models pass/fail before making changes, so new failures are attributable.

Workflow A: A Specific Model Was Given

1. Research the model (web search + provider docs):
   • Is it served on OpenRouter? Fetch https://openrouter.ai/api/v1/models and find the exact id (<vendor-slug>/<model-id>, watch for -preview, dated snapshots, dots vs hyphens).
   • Which other channels serve it (vendor native API, Aliyun DashScope, etc.) and what are their native model ids?
   • API differences: can thinking/reasoning be disabled or minimized? Any tool_choice, parallel_tool_calls, or parameter-schema quirks? Does the vendor's OpenAI-compatible endpoint differ from OpenRouter's behavior?
   • Agent suitability: tool call support is mandatory; note context window, latency, and cost.
2. Update modelPatch in packages/llms/src/utils.ts if the model needs new parameter handling. Model names are matched after normalizeModelName (lowercased, /-prefix, . and _ stripped) — check whether an existing family branch already covers it.
3. Add the model to both MODEL_GROUPS lists (website page and live test), newest first within its brand group. Add id overrides if the OpenRouter/Aliyun id differs from the display name.
4. Test availability carefully. Run the live suite and confirm the new model passes on every provider that serves it:

npm run test:live -w @page-agent/llms

A failure means the request/response shape is wrong for that model — fix modelPatch or the id override, don't shrug it off. If the model fails consistently on a provider, exclude it from that provider (see the deepseek-3.2 precedent in the test file) and document why in a comment. 5. Decide on BASELINE only if the model is a fast, cheap, strong-tool-call option; otherwise leave it out. 6. Run npm run typecheck and npm test.

Workflow B: No Model Given (Routine Maintenance)

1. Verify the existing list: run the live suite (see above). Investigate any model that newly fails — deprecated? renamed? provider dropped it?
2. Check for drift: re-verify OPENROUTER_ID_OVERRIDES against the current https://openrouter.ai/api/v1/models output; ids change when vendors promote previews to stable.
3. Scan for new models worth adding (web search for recent releases from the brands in MODEL_GROUPS, plus notable newcomers). A candidate must support tool calls via an OpenAI-compatible API.
4. Act by significance:
   • Minor updates (a preview id went stable, a small point-release replaces its predecessor): apply the change yourself following Workflow A steps 2–6, then present it to the user for judgment.
   • New models or removals: report findings and recommendations, let the user decide before editing.

Hand Off to the User

After your own tests pass, always remind the user to verify manually:

1. Tell them which line to put in the repo-root .env — give the exact model id per channel, including the OpenRouter id, e.g.:

LLM_BASE_URL="https://openrouter.ai/api/v1"
LLM_API_KEY="..."
LLM_MODEL_NAME="<exact-openrouter-id>"

2. Tell them to run the demo themselves: npm run dev:demo (serves on port 5174) and exercise the agent against a real page.

The live test only proves a single forced tool call round-trips; it is not an agent-quality eval. The user's manual run is the real acceptance test.