Ax Cpp AI

AxAI Providers For C++

This skill helps an agent write C++ code with the generated Ax package axllm. Use the generated package API, examples, and manifests; do not import TypeScript-only APIs unless you are editing the TypeScript package.

When To Use

• Create provider clients or normalize provider options.
• Choose a named deployment profile separately from the model ID served by that deployment.
• Attach renewable per-request credentials for expiring cloud tokens.
• Resolve structured-output modes from the selected profile and model.
• Choose between model-list routing, ordered failover, and adaptive operational routing.
• Route multimodal requests without flattening native images or files when the selected provider supports them.
• Use scripted transports for deterministic no-key examples.
• Use provider-api examples only when explicit provider credentials are available.

Package Facts

• Language: C++.
• Package: axllm.
• Package API docs: API.md and axir-api.json.
• Capability manifest: axir-capabilities.json.
• Runnable examples: examples/.
• Real network support: yes.
• Scripted no-key transport support: yes.
• Runtime profiles: javascript-quickjs, python-pyodide.

Core Pattern

#include "axllm/axllm.hpp"

auto llm = axllm::ai("openai", { {"apiKey", std::getenv("OPENAI_API_KEY")} });

Named Deployment Profiles

• The first ai / NewAI factory argument selects deployment behavior. The model option selects a model only inside that deployment; never infer request rules from a vendor-looking model ID.
• openai is the official OpenAI deployment. openai-compatible is the conservative custom-endpoint profile and requires an explicit base URL. Unknown profile names are errors.
• A Together-hosted DeepSeek model uses the together profile's URL, authentication, reasoning fields, and effort mapping. Native DeepSeek thinking fields apply only to the deepseek profile.
• Verified DeepSeek, Grok, Groq, Cerebras, and DeepInfra model rules default an omitted thinking level to logical max, mapped to the strongest documented deployment effort.
• Send none only where the selected deployment and model document reasoning disablement. Unsupported levels fail before network I/O; dynamic Hugging Face Router routes remain conservative.
• Structured output is an ordered model-aware capability: native, function, and json_object. Exact caller model metadata overrides the first matching profile rule, which overrides the profile default.
• An explicit unsupported structured-output mode fails before transport. structuredOutputs / structured_outputs remains the compatibility alias for native JSON Schema only.
• The exact Vertex google/gemma-4-26b-a4b-it-maas rule prefers json_object, excludes native schema, defaults thinking to max, writes nested enable_thinking, and extracts/replays reasoning_content. Unknown Vertex models stay conservative.
• Use named factories for Azure OpenAI, Cohere, DeepSeek, DeepSeek Responses, Mistral, Reka, Grok, routers, hosted inference, and configurable runtimes. Profile-only branded client constructors were removed.
• Retained client classes are transport/runtime boundaries: OpenAI-compatible Chat Completions, OpenAI Responses, Anthropic Messages, and Gemini GenerateContent. Build ordinary applications through the named factory.
• Provider descriptors and conformance fixtures are generated from the shared profile manifest. Do not add provider-name switches or cross-profile model normalization in a generated package.

Vertex And Prompt Caching

• Configure Gemini or Anthropic Vertex mode with projectId / project_id and region; optionally select a Vertex endpoint with endpointId / endpoint_id.
• Use credentialProvider / credential_provider for expiring Vertex and cloud tokens. It receives profile, operation, method, and URL on every attempt; its headers override static authentication.
• Credential callbacks cover chat, stream, embeddings, Responses, transcription, speech, and retries. Callback errors stop before transport, and completed 401/403 generation responses are not replayed automatically.
• Keep ADC and cloud SDK dependencies host-owned: obtain or refresh the token inside the callback. A required-auth profile accepts either a static key or the callback.
• Core resolves global, us, eu, and regional Vertex hosts. An explicit baseUrl / base_url takes precedence.
• OpenAI GPT-5.6 Chat explicit caching is opt-in through contextCache / context_cache or message/function cache flags. Use promptCacheKey / prompt_cache_key for stable affinity; sessionId / session_id is the fallback.
• Normalized usage separates uncached prompt, cache-read, and cache-creation tokens. get_model_cost / target equivalent uses the shared model catalog, including cache-write pricing and long-context thresholds.
• Start with the OpenAI prompt-caching and Vertex Gemini examples under examples/. Scripted AxAI fixtures verify routing without live credentials.

Routing And Balancing

• Use the multi-service router when a logical model key selects a configured service or concrete model. It combines model lists; it does not learn from outcomes.
• Use ProviderRouter for capability-based selection and optional media degradation. When the selected provider supports images, preserve every native image part with its payload, MIME type, detail level, cache and optimization hints, alt text, and ordering with surrounding text.
• Native files retain filename, MIME type, data, cache flags, extraction metadata, and order through provider/model selection and later conversation turns. Existing extracted text is used only when the selected provider cannot consume the file.
• For unsupported files, configure a file-to-text callback or choose degradation, skip, or error policy. An empty extraction result is valid; extraction failures stop before the provider request. Python, Go, and Java accept fileToText in router processing options; C++ exposes file_to_text, and Rust exposes with_file_to_text and with_processing.
• Inline PDF inputs should include filename, mimeType, and base64 data. See the public native-file-routing generation example for this language; it uses the ordinary generator and a real provider.
• Use the default AxBalancer for deterministic ordered/metric failover with its existing retry policy.
• Opt into AxBalancerAdaptiveStrategy only for operational routing among application-approved equivalent aliases. It learns transient reliability and successful latency, combines them with estimated cost and a deadline, and explores with Thompson sampling.
• Put centralized decision state in an AxBalancerStatsStore. The routing-event callback is best-effort analytics and observability, not a state replication mechanism.
• Shared stores require non-empty, unique, stable route keys. Use slices to isolate workflows, tenants, or traffic classes without putting prompts, responses, raw errors, or sensitive identifiers in keys or events.
• Adaptive balancing does not measure answer quality or semantically choose a model. Only group routes that the application already accepts as substitutes.
• Generated provider streams are incremental and closeable. Retry or failover is allowed only before the first content event; later failures surface without replay, and adaptive latency is recorded at the first chunk.
• Start with examples/adaptive_balancer_no_key for store/reducer syntax, then use the cataloged provider-backed adaptive-balancer example for a complete two-route setup.

Astra Session Work

Select gpt-6-astra through the ordinary OpenAI factory. The adapter chooses Responses automatically; existing model defaults are unchanged. Use low reasoning and standard processing. Portable minimal reasoning maps to low; none is rejected. EU residency does not support priority processing.

Keep applications on their generation, agent, and flow entrypoints. Declare only independent tools as background; ordinary and imported MCP tools stay blocking unless the application explicitly changes their declaration. A promise, thread, or MCP hint is not a background declaration. Set asyncMode to off for the ordinary tool loop; chat-only services retain that loop automatically.

Declared-background native agent tools retain the imported MCP schema, handler, namespace, and raw result. Discovery must expose the tool before the model can call it. Invalid arguments are corrected before handler execution; the responder waits for the incorporated result. Native calls appear in action logs and must not be repeated through actor code. Owned child agents inherit selected MCP clients at delegation. Parent stages keep their own clients; none or an empty namespace list passes no parent clients. Explicit child context wins over inherited context. Each run refreshes protocol modules without serializing live client handles into model requests. Cancellation propagates through a delegated child into its pending MCP tool; completed child work is not replayed. Imported MCP tools forward cancellation to context-aware transports, including built-in HTTP. Custom transports using the older send method receive cancellation checks before and after their call; noncooperative work may finish later and its result is discarded. Cancellation does not undo an external action or replay a request. MCP host policy applies to native background calls too. Configure authorizeToolCall in Python, Go, and Java client options, or set_tool_authorizer on C++ and Rust clients before exposing their tools. The callback receives the client and call metadata; returning false denies the call before a tool request is sent. Use shared application policy state when permissions must change during a run.

Register child agents before running the parent: add_child_agent(namespace, name, child) in Python/C++, AddChildAgent in Go, addChildAgent in Java, and with_child_agent in Rust. Registered children are available automatically as namespaced actor calls, such as team.researcher({question}). Calls use discovery, validation, and invocation accounting. Child invocation remains serialized on the owning run thread and owns a separate conversation. Retained callbacks reject calls after the run closes. Controls target paths such as root/team.researcher/executor. Child results return through the parent invocation log, and parent usage includes a children section.

Attach the language-native run controller through forward options for steering, reasoning changes, cancellation, and lifecycle events. Queued and applied are different states. HTTP applies updates at a response boundary; an optional host WebSocket enables native steering. Do not manage response IDs, socket messages, or tool-result submission in application code.

A provisional answer is not successful completion while started tools remain unresolved. Cancellation closes the session, reports unresolved call IDs, and retains unresolved started calls in tool traces and native agent action logs; it cannot undo an external action. Handlers may cooperate through the invocation cancellation context. Late results from noncooperative work must not change a closed run or trigger replay.

Java, C++, and Rust WebSocket adapters track activity when frames arrive. Consuming buffered events does not reactivate a completed response. When no response is active, steering is queued for the next response; an active successor can still receive native steering. Observe lifecycle timing instead of assuming native application.

All five session adapters validate completed raw arguments against the shared Core validator before invoking handlers, including local references, unions, nested schemas, additional properties, and numeric/string/array constraints. Raw schema patterns use shared flagless ECMAScript semantics, including UTF-16, lookarounds, named captures, and backreferences. Invalid arguments enter correction; step exhaustion fails the run.

Independent flow nodes use owned program and client workers. Built-in providers, routers, and balancers supply factories; custom implementations without them run the entire group serially and emit a flow_parallel_fallback trace. Rust does not require Send/Sync on the existing client trait. Rust nested flows and custom AxExecutableProgram implementations use execute_program; an optional AxOwnedProgramFactory constructs state on its worker. Workers deliver events and results to the owner, which merges successful results in plan order. On group failure, cancellation preserves completed diagnostics and discards late deliveries.

Use the provider-backed Astra examples under src/examples/cpp/generation/, short-agents/, and flows/. All-five generated parity remains under verification in the shared-session AxIR backlog; do not infer full agent, parallel-flow, or transport parity from these examples alone.

Relevant API Surface

• AxAI: axllm::ai, axllm::AxCancellationToken, axllm::AxAIServiceAbortedError, axllm::get_supported_ai_models, axllm::AxCredentialRequest, axllm::AxCredentialProvider, axllm::AIClient::owned_worker_factory, axllm::AxChatSession, axllm::AxStreamHandler, axllm::OpenAICompatibleClient, axllm::OpenAIResponsesClient, axllm::GoogleGeminiClient, axllm::AnthropicClient, axllm::AxUsageContext, axllm::AxUsageEvent, axllm::AxUsageObserver, axllm::set_usage_observer, axllm::AxRuntimeHooks, axllm::AxRateLimitInfo, axllm::AxRateLimiter, axllm::AxTracer, axllm::AxMeter, axllm::AxGlobals, axllm::set_rate_limiter, axllm::set_tracer, axllm::set_meter, axllm::AxBalancer, axllm::AxBalancerAdaptiveStrategy, axllm::AxBalancerStatsStore, axllm::AxInMemoryBalancerStatsStore, axllm::create_balancer_route_stats, axllm::update_balancer_route_stats, axllm::sample_balancer_route_health, axllm::MultiServiceRouter, axllm::ProviderRouter

Guardrails

• Start from package examples for exact native syntax before inventing a new call shape.
• Use provider-api examples only when the user explicitly has provider credentials available.
• Use no-key examples for deterministic local checks and provider request mapping.
• Treat AxIR as the source of generated package truth: if package docs disagree with source code, update the compiler and regenerate packages.
• Do not copy repo-maintainer skills from tools/*/skills/ into user packages.