Adding A Provider API Feature

Adding a provider API feature

Use this when exposing a new provider API capability through Pydantic AI — prompt caching, strict/structured tool calling, thinking/reasoning effort, service tier, safety settings, logprobs, cache breakpoints, and the like. The output is a change that is consistent with how sibling providers already expose the same concept, defaults deliberately, and gates support with a capability flag.

Not for: adding a new model id (that's add-new-model), a bug fix, or a refactor.

The one rule that prevents the most rework

Before designing anything, find the existing cross-provider abstraction that governs this capability and let its shape decide the API. Most "how should I expose this?" questions are already answered by an abstraction the codebase has — reaching for a new provider-specific knob when one exists is the single most common thing maintainers reject. When a feature routes through an existing abstraction, the abstraction's shape pre-decides the API surface, the opt-out, and often the default.

The tell that you skipped this: you find yourself listing 2-3 "options" for how a user controls the feature. If one of those options duplicates an existing cross-provider control, it isn't a real option — the existing abstraction wins.

Step 0 — Enumerate sibling precedent

For the capability you're adding, list how every provider that already has an analog exposes it, and name the governing existing abstraction. It is one of:

• a per-tool flag — ToolDefinition.strict: bool | None (tools.py), resolved in models/__init__.py::_customize_tool_def;
• a shared ModelSettings field — thinking, service_tier (settings.py), each with per-model resolvers mapping to native concepts;
• a provider-prefixed {Provider}ModelSettings field — anthropic_cache, openai_prompt_cache_key, groq_reasoning_effort;
• a message-stream marker — CachePoint in UserPromptPart.content (messages.py);
• a ModelProfile capability flag — openai_supports_strict_tool_definition, bedrock_supports_prompt_caching.

Read the actual sibling implementations and any review threads on the PRs that added them (gh pr view <n> --comments). Contributors who skipped this got redirected: raw google_tool_config → use strict (#5366); string-prefix model detection → use a profile flag (#4604). Only open a design fork if no existing abstraction covers the capability.

Step 1 — Pick the API shape

1. An existing cross-provider abstraction covers it → reuse it. Add a provider mapping (a _translate_* resolver, a JsonSchemaTransformer subclass, a CachePoint translation). Do not add a provider-prefixed knob for something the shared abstraction already expresses.
2. No shared abstraction, but ≥3 providers now have the concept → promote to a shared ModelSettings field with a deliberately narrow common vocabulary and per-model resolvers, keeping per-provider fields underneath as precedence-winning escape hatches (the service_tier promotion, #4926). Don't delete provider knobs; deprecate only genuinely-misnamed ones with a TODO(v3).
3. The provider's native values can't be expressed by the shared enum (family-disjoint value sets, platform-only request shaping) → a {provider}_* knob is justified, and it coexists with and outranks the unified field (groq_reasoning_effort, #5797).
4. Choose the locus by "can the user naturally point at it?": a boundary inside the message stream → a marker (CachePoint); a structural region (system prompt, tool defs, whole-request setting) → a setting (#3363).
5. Wire it into every request path the provider has (chat and responses APIs); put a setting shared across a provider's APIs on the base settings class (#3678).
6. Type it. Reuse the provider SDK's own types where they exist; type knobs as Literal, never extra_body or untyped **kwargs (models/AGENTS.md; "kwargs are a big no no… I'd rather be repetitive but type safe" — DouweM, #3457).

Step 2 — Default: default-on (silent) vs opt-in

Default the feature on only when enabling it cannot change observable behavior and cannot cost the user — a pure, backward-compatible improvement (caching only lowers cost; a validation mode that needs no schema rewrite and can't reject a previously-valid request). A backward-compatible, purely-better default is welcome and needs no opt-in.

Keep it opt-in when it:

• changes observable output or wire behavior;
• is a preview feature (provider may change semantics);
• can raise cost — and choose the default value so a shared field never silently upgrades anyone to a pricier tier (service_tier 'auto' vs 'default', #4926);
• can hit provider limits at scale (auto-promoting strict silently broke agents with >20 tools; reverted in #5580);
• applies lossy schema rewrites (OpenAI/Anthropic strict transforms).

When the default isn't obvious, decide it with a live probe, not an opinion (#5897 flipped a default on 0/5 → 5/5 recovery). Beware "automatic" language — verify whether it means a Pydantic AI default or just provider-side management of an opt-in feature.

Step 3 — Capability gating

Detect support with a provider-prefixed ModelProfile flag set in Provider.model_profile() — never inline isinstance/model-name checks (profiles/AGENTS.md). Gate at the layer(s) that actually vary:

• model/family → profile flag (google_supports_strict_tool_definition, bedrock_supports_prompt_caching);
• per-schema → the JsonSchemaTransformer.is_strict_compatible signal (default conservative unless the mode needs no rewrites);
• SDK version → probe the SDK shape and degrade with a UserWarning (#5580, botocore strict param);
• unknowable client-side → defer to the runtime API error.

Unsupported → silently ignore the setting (best-effort so as many requests as possible succeed), documented in the docstring — never hard-error. Conflicting user settings → UserWarning, not UserError. Profiles are layered: developer-keyed base + thin provider overlay resolved per family, with the provider overlay layering last (#5934, #6231).

Step 4 — Tests

Live-recorded wire-contract cassette asserting the exact outbound body (mode/field is sent for supported models, absent/ignored for unsupported), plus a test exercising the new setting. Prefer case-based parametrized VCR over mocks; a unit test is still right for asserting an internal request shape a cassette matcher wouldn't catch. This is also where a review bot's "this will fail" claim gets refuted with a recorded cassette.

Step 5 — Docs & skills

The new public symbol's docstring lists which providers support it and how each interprets the value. Add/refresh the docs/**.md section and any sibling docstrings that now under-claim ("only OpenAI" → the full provider list). Describe the mechanism only as far as the provider documents it — don't assert a mechanism a provider's docs leave unstated. Update the relevant agent skill.

ModelSettings' Supported by: lists are enforced: tests/models/test_model_settings_support.py probes each model class's outgoing request and asserts every list names exactly the classes that send the field. Forwarding a new setting means editing its list, and a new Model class means adding a Case there. tool_choice and thinking are exempt via HAND_MAINTAINED and stay hand-maintained.

Recurring maintainer principles (quoted)

1. Reuse the cross-provider abstraction over a provider knob — "this maps to what the Anthropic and OpenAI APIs call strict… already represented on ToolDefinition… a more complete, consistent, 'doesn't require the user to do something special' implementation would be to automatically use this mode." (#5366)
2. Promote to a shared setting once several providers have the concept; keep per-provider overrides underneath. (#4926)
3. Best-effort — silently ignore unsupported settings, don't error. "we typically do a 'best effort' so that as many requests as possible succeed." (#3438)
4. Hide provider complexity — the feature should be useful to people who don't want to become experts in that provider's limitations.
5. Capability facts belong on ModelProfile flags, layered base + overlay. (#5934)
6. Type safety over repetition; no untyped kwargs. (#3457)
7. Put shared fields on the base settings class; cover every API surface. (#3678)
8. Verify against the real API; defer validation to runtime. "If they're allowed by the SDK types and we can try it out and it doesn't fail, I'm fine with it." (#3678)
9. Name the eventual unification even when deferring it — a {provider}_* knob today can note the future Caching/Thinking-style capability it should fold into. (#4604)

Precedent map