Add Hosted Key

Adding Hosted Key Support to a Tool

When a tool has hosted key support, Sim provides its own API key if the user hasn't configured one (via BYOK or env var). Usage is metered and billed to the workspace.

Overview

Step 1: Register the BYOK Provider ID

Add the new provider to the BYOKProviderId union in tools/types.ts:

export type BYOKProviderId =
  | 'openai'
  | 'anthropic'
  // ...existing providers
  | 'your_service'

Then add the same provider id to the byokProviderIdSchema enum in lib/api/contracts/byok-keys.ts (this is what the byok-keys route validates against):

export const byokProviderIdSchema = z.enum([
  'openai',
  'anthropic',
  // ...existing providers
  'your_service',
])

Step 2: Research the API's Pricing Model and Rate Limits

Before writing any getCost or rateLimit code, look up the service's official documentation for both pricing and rate limits. You need to understand:

Pricing

1. How the API charges — per request, per credit, per token, per step, per minute, etc.
2. Whether the API reports cost in its response — look for fields like creditsUsed, costDollars, tokensUsed, or similar in the response body or headers
3. Whether cost varies by endpoint/options — some APIs charge more for certain features (e.g., Firecrawl charges 1 credit/page base but +4 for JSON format, +4 for enhanced mode)
4. The dollar-per-unit rate — what each credit/token/unit costs in dollars on our plan

Rate Limits

1. What rate limits the API enforces — requests per minute/second, tokens per minute, concurrent requests, etc.
2. Whether limits vary by plan tier — free vs paid vs enterprise often have different ceilings
3. Whether limits are per-key or per-account — determines whether adding more hosted keys actually increases total throughput
4. What the API returns when rate limited — HTTP 429, Retry-After header, error body format, etc.
5. Whether there are multiple dimensions — some APIs limit both requests/min AND tokens/min independently

Search the API's docs/pricing page (use WebSearch/WebFetch). Capture the pricing model as a comment in getCost so future maintainers know the source of truth.

Setting Our Rate Limits

Our rate limiter (lib/core/rate-limiter/hosted-key/) uses a token-bucket algorithm applied per billing actor (workspace). It supports two modes:

• per_request — simple; just requestsPerMinute. Good when the API charges flat per-request or cost doesn't vary much.
• custom — requestsPerMinute plus additional dimensions (e.g., tokens, search_units). Each dimension has its own limitPerMinute and an extractUsage function that reads actual usage from the response. Use when the API charges on a variable metric (tokens, credits) and you want to cap that metric too.

When choosing values for requestsPerMinute and any dimension limits:

• Stay well below the API's per-key limit — our keys are shared across all workspaces. If the API allows 60 RPM per key and we have 3 keys, the global ceiling is ~180 RPM. Set the per-workspace limit low enough (e.g., 20-60 RPM) that many workspaces can coexist without collectively hitting the API's ceiling.
• Account for key pooling — our round-robin distributes requests across N hosted keys, so the effective API-side rate per key is (total requests) / N. But per-workspace limits are enforced before key selection, so they apply regardless of key count.
• Prefer conservative defaults — it's easy to raise limits later but hard to claw back after users depend on high throughput.

Step 3: Add hosting Config to the Tool

Add a hosting object to the tool's ToolConfig. This tells the execution layer how to acquire hosted keys, calculate cost, and rate-limit.

hosting: {
  envKeyPrefix: 'YOUR_SERVICE_API_KEY',
  apiKeyParam: 'apiKey',
  byokProviderId: 'your_service',
  pricing: {
    type: 'custom',
    getCost: (_params, output) => {
      if (output.creditsUsed == null) {
        throw new Error('Response missing creditsUsed field')
      }
      const creditsUsed = output.creditsUsed as number
      const cost = creditsUsed * 0.001 // dollars per credit
      return { cost, metadata: { creditsUsed } }
    },
  },
  rateLimit: {
    mode: 'per_request',
    requestsPerMinute: 100,
  },
},

Hosted Key Env Var Convention

Keys use a numbered naming pattern driven by a count env var:

YOUR_SERVICE_API_KEY_COUNT=3
YOUR_SERVICE_API_KEY_1=sk-...
YOUR_SERVICE_API_KEY_2=sk-...
YOUR_SERVICE_API_KEY_3=sk-...

The envKeyPrefix value (YOUR_SERVICE_API_KEY) determines which env vars are read at runtime. Adding more keys only requires bumping the count and adding the new env var.

Pricing: Prefer API-Reported Cost

Always prefer using cost data returned by the API (e.g., creditsUsed, costDollars). This is the most accurate because it accounts for variable pricing tiers, feature modifiers, and plan-level discounts.

When the API reports cost — use it directly and throw if missing:

pricing: {
  type: 'custom',
  getCost: (params, output) => {
    if (output.creditsUsed == null) {
      throw new Error('Response missing creditsUsed field')
    }
    // $0.001 per credit — from https://example.com/pricing
    const cost = (output.creditsUsed as number) * 0.001
    return { cost, metadata: { creditsUsed: output.creditsUsed } }
  },
},

When the API does NOT report cost — compute it from params/output based on the pricing docs, but still validate the data you depend on:

pricing: {
  type: 'custom',
  getCost: (params, output) => {
    if (!Array.isArray(output.searchResults)) {
      throw new Error('Response missing searchResults, cannot determine cost')
    }
    // Serper: 1 credit for <=10 results, 2 credits for >10 — from https://serper.dev/pricing
    const credits = Number(params.num) > 10 ? 2 : 1
    return { cost: credits * 0.001, metadata: { credits } }
  },
},

getCost must always throw if it cannot determine cost. Never silently fall back to a default — this would hide billing inaccuracies.

Capturing Cost Data from the API

If the API returns cost info, capture it in transformResponse so getCost can read it from the output:

transformResponse: async (response: Response) => {
  const data = await response.json()
  return {
    success: true,
    output: {
      results: data.results,
      creditsUsed: data.creditsUsed,  // pass through for getCost
    },
  }
},

For async/polling tools, capture it in postProcess when the job completes:

if (jobData.status === 'completed') {
  result.output = {
    data: jobData.data,
    creditsUsed: jobData.creditsUsed,
  }
}

Step 4: Hide the API Key Field When Hosted

In the block config (blocks/blocks/{service}.ts), add hideWhenHosted: true to the API key subblock. This hides the field on hosted Sim since the platform provides the key:

{
  id: 'apiKey',
  title: 'API Key',
  type: 'short-input',
  placeholder: 'Enter your API key',
  password: true,
  required: true,
  hideWhenHosted: true,
},

The visibility is controlled by isSubBlockHidden() in lib/workflows/subblocks/visibility.ts, which checks both the isHosted feature flag (hideWhenHosted) and optional env var conditions (hideWhenEnvSet).

Excluding Specific Operations from Hosted Key Support

When a block has multiple operations but some operations should not use a hosted key (e.g., the underlying API is deprecated, unsupported, or too expensive), use the duplicate apiKey subblock pattern:

1. Remove the hosting config from the tool definition for that operation — it must not have a hosting object at all.
2. Duplicate the apiKey subblock in the block config with opposing conditions:

// API Key — hidden when hosted for operations with hosted key support
{
  id: 'apiKey',
  title: 'API Key',
  type: 'short-input',
  placeholder: 'Enter your API key',
  password: true,
  required: true,
  hideWhenHosted: true,
  condition: { field: 'operation', value: 'unsupported_op', not: true },
},
// API Key — always visible for unsupported_op (no hosted key support)
{
  id: 'apiKey',
  title: 'API Key',
  type: 'short-input',
  placeholder: 'Enter your API key',
  password: true,
  required: true,
  condition: { field: 'operation', value: 'unsupported_op' },
},

Both subblocks share the same id: 'apiKey', so the same value flows to the tool. The conditions ensure only one is visible at a time. The first has hideWhenHosted: true and shows for all hosted operations; the second has no hideWhenHosted and shows only for the excluded operation — meaning users must always provide their own key for that operation.

To exclude multiple operations, use an array: { field: 'operation', value: ['op_a', 'op_b'] }.

Reference implementation: blocks/blocks/google_maps.ts — speed_limits (deprecated Roads API) is excluded from hosting with the duplicate apiKey pair.

Step 5: Add to the BYOK Settings UI

Add an entry to the PROVIDERS array in the BYOK settings component so users can bring their own key. You need the service icon from components/icons.tsx:

{
  id: 'your_service',
  name: 'Your Service',
  icon: YourServiceIcon,
  description: 'What this service does',
  placeholder: 'Enter your API key',
},

Step 6: Summarize Pricing and Throttling Comparison

After all code changes are complete, output a detailed summary to the user covering:

What to include

1. API's pricing model — how the service charges (per token, per credit, per request, etc.), the specific rates found in docs, and whether the API reports cost in responses.
2. Our getCost approach — how we calculate cost, what fields we depend on, and any assumptions or estimates (especially when the API doesn't report exact dollar cost).
3. API's rate limits — the documented limits (RPM, TPM, concurrent, etc.), which plan tier they apply to, and whether they're per-key or per-account.
4. Our rateLimit config — what we set for requestsPerMinute (and dimensions if custom mode), why we chose those values, and how they compare to the API's limits.
5. Key pooling impact — how many hosted keys we expect, and how round-robin distribution affects the effective per-key rate at the API.
6. Gaps or risks — anything the API charges for that we don't meter, rate limit dimensions we chose not to enforce, or pricing that may be inaccurate due to variable model/tier costs.

Format

Present this as a structured summary with clear headings. Example:

### Pricing
- **API charges**: $X per 1M tokens (input), $Y per 1M tokens (output) — varies by model
- **Response reports cost?**: No — only token counts in `usage` field
- **Our getCost**: Estimates cost at $Z per 1M total tokens based on median model pricing
- **Risk**: Actual cost varies by model; our estimate may over/undercharge for cheap/expensive models

### Throttling
- **API limits**: 300 RPM per key (paid tier), 60 RPM (free tier)
- **Per-key or per-account**: Per key — more keys = more throughput
- **Our config**: 60 RPM per workspace (per_request mode)
- **With N keys**: Effective per-key rate is (total RPM across workspaces) / N
- **Headroom**: Comfortable — even 10 active workspaces at full rate = 600 RPM / 3 keys = 200 RPM per key, under the 300 RPM API limit

This summary helps reviewers verify that the pricing and rate limiting are well-calibrated and surfaces any risks that need monitoring.

Checklist

• Provider added to BYOKProviderId in tools/types.ts
• Provider added to byokProviderIdSchema enum in lib/api/contracts/byok-keys.ts
• API pricing docs researched — understand per-unit cost and whether the API reports cost in responses
• API rate limits researched — understand RPM/TPM limits, per-key vs per-account, and plan tiers
• hosting config added to the tool with envKeyPrefix, apiKeyParam, byokProviderId, pricing, and rateLimit
• getCost throws if required cost data is missing from the response
• Cost data captured in transformResponse or postProcess if API provides it
• hideWhenHosted: true added to the API key subblock in the block config
• Provider entry added to the BYOK settings UI with icon and description
• Env vars documented: {PREFIX}_COUNT and {PREFIX}_1..N
• Pricing and throttling summary provided to reviewer