# Coral Create Source Spec > Create or update a Coral source spec YAML for a custom HTTP API or local dataset. Use when authoring a standalone source for `coral source add --file`, or when adapting that spec into a Coral repo source under `sources/core` or `sources/community`. - Skill: `withcoral/coral-create-source-spec` (Agent Skill, multi-file: 3 files) - Install (CLI): `npx skillmds@latest add withcoral/coral-create-source-spec` - Raw SKILL.md: https://api.skillmd.com/api/skills/withcoral/coral-create-source-spec/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - Author: withcoral (https://skillmd.com/u/withcoral) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/withcoral/coral-create-source-spec --- # Coral Create Source Spec Use this skill when the task is to author or repair a Coral source spec. ## Goal Produce a valid, queryable Coral source spec that works with: - `coral source lint ` - `coral source add --file ` - `coral source test ` - `coral sql` - `coral.tables` and `coral.columns` - `coral.inputs` for source variables and secrets ## Default Mode Default to standalone source authoring for external developers. That means: - create a YAML source spec file - lint it early with `coral source lint ` - add it to Coral with `coral source add --file ` when you need to exercise it as a source - validate by querying it - iterate until the shape is correct Only switch to Coral repo layout when the user is explicitly editing the Coral repo. ## Output Modes - External authoring: - create a standalone source spec such as `./my-source.yaml` - validate structure with `coral source lint ./my-source.yaml` - load it with `coral source add --file ./my-source.yaml` when you need to query it through Coral - Coral repo contribution: - write community source specs to `sources/community//manifest.yaml` - write core source specs to `sources/core//manifest.yaml` only when the user is intentionally changing bundled core sources - add representative `test_queries` for a basic smoke/connection check of the source - validate with `coral source test ` and repo checks ## Workflow 1. Read the provider API docs or inspect the local dataset. 2. Start with one small table and a few columns. 3. Define: - source metadata - backend - base URL or file location - auth - variables and secrets - credential retrieval methods for secrets, including OAuth when the provider supports browser-based setup - if authoring a separate `kind: identity` manifest, identity setup inputs and OAuth method config - tables - table functions for source-scoped parameterized endpoints - filters - response extraction - pagination - typed columns 4. Lint the source spec: - `coral source lint ` 5. Validate the source in the right mode: - standalone specs: `coral source add --file ` and inspect with `coral sql` - `coral source add` is non-interactive by default: each input `key` is read from the matching environment variable. Export required variables and secrets before running, or pass `--interactive` to be prompted. - for OAuth credential methods, run `coral source add --interactive --file ` with no environment value for the target secret so Coral offers the authored credential choices - repo sources or already-named sources: `coral source test ` 6. Inspect the exposed shape: - inspect `coral.tables` for visible tables, descriptions, guides, and required filters; keep metadata queries bounded with `LIMIT`/`OFFSET` - inspect `coral.table_functions` for source-scoped functions, descriptions, guides, arguments, result columns, kind, and search limits - inspect `coral.columns` for canonical column metadata, including `is_virtual` and `is_required_filter`; filter by one table or page large column sets - inspect `coral.filters` for normalized table filter names, types, modes, required flags, and descriptions - inspect `coral.inputs` to verify variables, secrets, defaults, hints, and required flags 7. Query representative tables with `coral sql`. 8. If you are relying on `coral source test`, make sure `test_queries` gives you a basic smoke/connection check for the source. 9. Refine the spec and repeat. ## Authoring Rules - Start small and expand table coverage incrementally. - Use the source manifest schema as both inspiration for authoring and validation of structure: https://github.com/withcoral/coral/blob/main/crates/coral-spec/src/schema/source_manifest.schema.json - Use source variables for non-secret configuration. - Use source secrets for credentials. - For OAuth-backed services, model setup with `inputs..credential.methods[]` using `type: oauth`; keep the runtime `auth` or request header pointing at the same secret input. - OAuth credential methods support device-code, authorization-code, and client-credentials flow. For authorization-code flow, set `flow.type: authorization_code`, set `flow.pkce` explicitly to `required` or `disabled`, use a loopback `http://127.0.0.1` or `http://localhost` redirect URI, choose `redirect_uri_port_mode: random` for provider apps or Dynamic Client Registration endpoints that allow variable localhost ports, and choose `fixed` only when users can register the exact non-zero redirect URI. The CLI also accepts the final loopback redirect URL pasted into the terminal when the browser cannot reach the machine running Coral, so do not reject authorization-code OAuth solely because users may run Coral over SSH, in a VM, or in another split-browser environment. For device-code flow, declare `flow.type: device_code`, `endpoints.device_authorization_url`, `endpoints.token_url`, and either a public client ID or `client.dynamic_registration`; omit redirect URI fields and do not declare a static client secret. For client-credentials flow, declare `flow.type: client_credentials`, `endpoints.token_url`, `client.id.input`, `client.secret.input`, and `client.secret.transport`; omit redirect URI fields, authorization endpoints, device authorization endpoints, PKCE, and Dynamic Client Registration. - When an authorization-code or device-code provider supports OAuth Dynamic Client Registration, prefer `client.dynamic_registration.registration_url` over hard-coding a shared public client ID. Coral always registers a native client; source specs cannot configure a `web` Dynamic Client Registration application type. Set `token_endpoint_auth_method` to `none`, `client_secret_basic`, or `client_secret_post` to match the registered client type. Set `request_refresh_token_grant: true` only when the provider supports registering clients for refresh-token grants. For OAuth-protected Streamable HTTP MCP servers, set `oauth.resource` to the MCP server URL when the provider requires resource indicators. - OAuth URL fields may template declared `kind: variable` inputs with `{{input.KEY}}` for non-secret endpoint components such as tenant IDs or domains. This includes `endpoints.*`, `resource`, and `client.dynamic_registration.registration_url`. Do not reference secret inputs, filters, function arguments, state, or inline defaults from OAuth URLs. - If a provider also supports manually pasted tokens, include a `type: source_config` fallback after the OAuth method. When the provider's token endpoint requires client authentication with a client secret, prompt for both OAuth client values: declare `client.id.input`, `client.secret.input`, and `client.secret.transport` (`basic_auth` or `request_body`). - Do not add top-level source inputs solely for OAuth client credentials; `client.id.input` and `client.secret.input` are collected during OAuth setup. - In a `kind: identity` manifest, put provider `issuer`, identity `type`, setup inputs, OAuth URLs, client IDs, and client secrets on the identity spec itself. Identity OAuth `client.id` may use `default`, a variable `input`, or both; `client.secret.input` must reference a declared secret input. OAuth URL templates may reference only declared variable inputs. Identity inputs must not declare source credential methods. - Each credential method accepts optional `label`, `description`, and `hint` fields, surfaced during interactive install and in the generated source docs. When an input offers more than one method, put the how-to-get-it guidance in each method's `hint` (rendered next to that method's fields) instead of in one long input-level `hint`, and scope each hint to the inputs that method collects. - For short-lived OAuth access tokens, make sure authorization-code or device-code methods can obtain refresh tokens when the provider supports them, and document any scopes, consent prompts, or client settings required for refresh-token issuance. Client-credentials methods re-mint access tokens from stored client metadata when the provider returns usable `expires_in` metadata; without it, users must reconnect after the token expires. If another OAuth method will not issue refresh tokens, call out that users must reconnect when access tokens expire unless the source has another supported long-lived credential path. - Keep table and table-function names stable, SQL-friendly, and unique within the source's case-insensitive relation namespace. Prefer plain `snake_case` table names. Table-function names must start with an ASCII letter or underscore and then use only ASCII letters, numbers, or underscores. - Mark filters as required only when the API truly requires them. - Use default table functions for parameterized non-retrieval operations, such as scoped child collections, time-range logs, metrics queries, or detail operations that do not map cleanly to a stable table. - Use `kind: search` table functions for provider endpoints that accept query text and return ranked candidates. - Set a table-function arg `type` when the provider argument is not string-shaped; omitted arg types default to `Utf8`. - Do not model provider search as a table filter. Use `mode: contains` only for ordinary provider-side substring filters. Provider-ranked retrieval belongs in a `kind: search` function. - Include `search_limits` on every `kind: search` function and expose stable result identifiers for follow-up detail queries. - Prefer explicit pagination when the API shape is known. - Verify pagination with actual row fetches, not only `COUNT(*)`. - Add or update `test_queries` when you want `coral source test` to perform a basic smoke/connection check. ## Metadata UX Rules Use these rules for top-level source metadata so source discovery and setup are consistent. ### `description` - Start with `Query ...`. - Make the first sentence capability-first: list the key entities users can query. - Preferred template: - `Query from ().` - Keep `description` focused on data coverage, not setup steps. - Do not use vague phrasing such as: - `REST API v3` - `OpenAPI provider` - `... and more` - Move auth/setup/permission details to input hints, not description text. ### Input hints (`inputs..hint`) Each hint should tell the user: - what value is expected - how to obtain it - minimum scope/permission guidance - one concrete format/example when useful Specific guidance: - For URL/base inputs: - say what the default means - include at least one concrete example - include self-hosted guidance when supported - For secrets: - name the exact credential type (API key, PAT, application key, etc.) - include format constraints when relevant (for example, token prefixes) - include least-privilege scope guidance - For OAuth methods: - use a user-facing `label` such as `Connect with GitHub` - keep `description` to a short one-line blurb; put the setup detail in the method's `hint` - in the method's `hint`, list the required OAuth scopes and explain whether users must register a fixed loopback redirect URI or provide their own OAuth client ID/secret; when the method collects `client.id.input`/`client.secret.input`, say where to obtain those values - for authorization-code flow, note in the method's `hint` that users can paste the final localhost redirect URL into the terminal if their browser cannot reach Coral's loopback listener directly - when a secret offers multiple methods, scope each method's `hint` to the inputs that method collects instead of writing one broad input-level hint that mixes guidance for every method - For derived secrets (for example Basic auth blobs): - include a short shell example (for example a Base64 command) - Prefer stable documentation links. - Use official docs links and stable settings pages. - Avoid brittle click-path instructions as the primary guidance. Keep hints concise and directly actionable. ## Validation Loop Use this loop during authoring: ```sh # Export any required inputs first (key matches the input `key` in the spec), # or pass --interactive to be prompted. coral source lint ./my-source.yaml coral source add --file ./my-source.yaml coral source test my_source coral sql "SELECT catalog_name, schema_name, table_name, description, required_filters FROM coral.tables WHERE schema_name = 'my_source' OR catalog_name = 'my_source' ORDER BY catalog_name, schema_name, table_name LIMIT 50 OFFSET 0" coral sql "SELECT function_name, kind, arguments_json, result_columns_json, search_limits_json FROM coral.table_functions WHERE schema_name = 'my_source' ORDER BY function_name LIMIT 50 OFFSET 0" coral sql "SELECT table_name, filter_name, filter_mode, is_required, data_type, description FROM coral.filters WHERE schema_name = 'my_source' OR catalog_name = 'my_source' ORDER BY table_name, filter_name LIMIT 100 OFFSET 0" coral sql "SELECT table_name, column_name, data_type, is_virtual, is_required_filter, filter_mode, description FROM coral.columns WHERE schema_name = 'my_source' OR catalog_name = 'my_source' ORDER BY table_name, ordinal_position LIMIT 100 OFFSET 0" coral sql "SELECT catalog_name, key, kind, value, default_value, hint, required, is_set FROM coral.inputs WHERE schema_name = 'my_source' OR catalog_name = 'my_source' ORDER BY key" ``` Each `WHERE` clause matches on both qualifiers because a source is addressed either by schema or by catalog. Schema-addressed sources carry the source name in `schema_name` and leave `catalog_name` empty; database sources put the source name in `catalog_name`, and their `coral.inputs` rows have an empty `schema_name`. Filtering on `schema_name` alone returns zero rows for a database source without reporting an error, which reads as "this source declares nothing". For repo sources or already-named sources, add `test_queries` for a basic smoke/connection check and run: ```sh coral source test my_source ``` Then run targeted table queries until the source behaves correctly. ## HTTP Sources For HTTP-backed sources: - define `backend: http` - define `base_url` - define auth headers or other runtime auth fields - define `credential.methods` on secret inputs when setup should offer OAuth or another retrieval choice - define request path, query, and body only where needed - define source-scoped table functions for provider-native operations that require invocation arguments - define response `rows_path` - define pagination explicitly when the provider pattern is known - define typed columns - add `test_queries` once you know which simple query or queries should confirm the source basically works Read `references/http-source-checklist.md` when you need table-shape and pagination guidance. If your HTTP source uses an Authorization header with a prefix (e.g. `Authorization: Bearer `), use a secret input for the raw token and define the header with `from: bearer`: ```yaml inputs: FOOBAR_API_TOKEN: kind: secret hint: Bearer token for the Foobar API. auth: type: HeaderAuth headers: - name: Authorization from: bearer key: FOOBAR_API_TOKEN ``` For an OAuth-backed HTTP source, add the retrieval method to that same secret input: ```yaml inputs: FOOBAR_API_TOKEN: kind: secret hint: Connect with Foobar OAuth or paste a token with read access. credential: methods: - type: oauth label: Connect with Foobar description: Open a browser and authorize Coral to read Foobar data. hint: | Signs you in through Foobar and requests the `read` scope. To use your own app, set FOOBAR_OAUTH_CLIENT_ID to its Client ID. oauth: flow: type: authorization_code pkce: required redirect_uri: http://127.0.0.1:0/oauth/callback redirect_uri_port_mode: random endpoints: authorization_url: https://foobar.example.com/oauth/authorize token_url: https://foobar.example.com/oauth/token client: id: input: FOOBAR_OAUTH_CLIENT_ID scopes: scope: delimiter: space values: - read - type: source_config label: Paste token hint: Paste a Foobar API token with read access to the data you query. auth: type: HeaderAuth headers: - name: Authorization from: bearer key: FOOBAR_API_TOKEN ``` When a provider accepts either a full pasted API-key header or an OAuth access token, declare both credential inputs as optional secrets, then use `from: one_of` and put the complete header value first, followed by a `from: bearer` OAuth fallback: ```yaml inputs: FOOBAR_API_KEY: kind: secret required: false FOOBAR_OAUTH_ACCESS_TOKEN: kind: secret required: false auth: type: HeaderAuth headers: - name: Authorization from: one_of values: - from: input key: FOOBAR_API_KEY - from: bearer key: FOOBAR_OAUTH_ACCESS_TOKEN ``` ## Local Data Sources For local file-backed sources: - define the file backend - define the source location - define file selection patterns if applicable - define typed columns ## Deliverable Report: - source spec path - lint / add / test commands used - validation commands run - assumptions made - blocked or unverified endpoints